Virtual neuro-psychological testing protocol

ABSTRACT

The present invention software driven protocol for managing a virtual clinical neuro-psychological testing program delivers, to individual or small groups of certified Neuro-psychology professionals, an interface protocol for clinically monitoring and managing the neuro-psychological needs of very large numbers of individual patients. The preferred approach to such monitoring and testing, is within the scope of normal and routine activities. These patients are most often in need of regular testing, screening, and monitoring and, only occasionally, of referral or intervention. The protocol provides a majority of patients, with regular testing, screening, and monitoring services on a “virtual-out-patient” basis by interactively delivering individually configured batteries of tests to each patient; via the Internet. These tests include normal activity observation and monitoring. The delivery and respectively corresponding acceptance of the testing batteries is a substantially automatically scheduled feature of the protocol.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national stage application of PCT Application No.PCT/IL01/00842 which was filed on Sep. 6, 2001 and published in Englishon Mar. 14, 2002 as International Publication No. WO 02/19889 (the“International Application”). This application claims priority from theInternational Application pursuant to 35 U.S.C. § 365. The presentapplication also claims priority under 35 U.S.C. § 119 from IsraeliPatent Application No. IL 138322, filed Sep. 7, 2000.

BACKGROUND OF THE INVENTION

Ideally, in the field of neuro-psychology, a trained, fully qualifiedexpert should have the opportunity to spend a substantial amount of timewith each client. This time should include a significant amount ofobservation and testing, on a one-on-one basis. Unfortunately, thereexists a chronic shortage of such professionals and, therefore, theamount of time that can be spent with each client is very limited. Inaddition, there is a shortage of testing materials whereby aprofessional can monitor the testing of each client without beingphysically present but nonetheless in control of the testing.

There are, in existence and in use, many types of standardizedneuro-psychological tests carried out through the medium of computers;but these have a significant drawback of being generally rather static.These are so described, not simply because the contents are monotonous,but, also, because the method of application is rather rigidlyimplemented. Each test needs to be substantially completed beforeproceeding to the next without any consideration of the nature of theanswers or the reactions of the client.

Furthermore, when these tests are arranged into batteries of tests, thebatteries become even more static, if not positively unwieldy. Thereason for this is that setting up batteries of tests is done in astandardized manner and, only after the battery of tests aresubstantially completed by a client, does the neuro-psychologist reviewthe results.

It should also be explained that a testing procedure carried out underthe direct supervision of an experienced and expert professionalneuro-psychologist is not carried out in the static manner described,but with continuous monitoring. If the professional tester reachesconclusions prior to the end of a test, the client will be redirected tofurther tests, rather than be allowed to proceed with a rigid regime oftesting. It would, therefore, be of substantial advantage if thereexisted a methodology that could construct the necessary batteries oftests, in the same manner and using the same approach, that would beutilized by a group of well qualified and expert neuro-psychologists.

Furthermore, there are a number of factors inhibiting the growth andimprovements of neuro-psychological testing, most particularly becausethe validation of tests is extremely costly, very slow and, perhaps mostimportant of all, very time consuming. It would be most useful if newtests and new ways of testing could easily be validated.

The problems associated with validating neuro-psychological tests arethat validation must explain a correlation to known instructions, mustprovide results with a usably narrow distribution and, hence, theability to decide the result.

It is not always obvious that tests are a structured means of capturinga description of how an individual performs in a given set of controlledand observable circumstances. This statement underlines the basicproblem. Neuro-psychological experts are not able to invest sufficienttime with each client to be able to fully perceive the full and detailednature of each problem. Ideally, there is an important need tounderstand, precisely, the nature of the client and his special needs.

Therefore, it would be of significant importance and advantage, if meansexisted to embody the metrics from validated tests together with theexpertise of time-constrained experts and bring such formed anddeveloped tests to bear in a field so desperately in need ofadvancement. It would be of even greater significance, if all this couldbe brought to bear utilizing normal routine activities of each client.

A specific objective of every neuro-psychologist is to help clients withtheir normal routine activities. Because such professionals do not havethe time available to, the issue of a new and highly useful methodologyof testing by monitoring normal routine activities on an ongoing basis,must come to the fore.

High level certification as a neuro-psychology professional, such as aneurologist, involves a long and complex course of study andapprenticeship. Ultimately this course of study is only successfullycompleted by a small number of individuals. Few patients are fortunateenough to receive the full benefits that may be provided by thesecertified professionals, essentially because there are to few suchprofessionals available, and their time is over subscribed by the greatnumbers of patients needing their help.

Today, neuro-psychology professional services are restricted to foursubstantially small sub-populations, of the greater population ofpatients, who could benefit from proper care. These sub-populations are:persons needing screening, severely disabled persons such as accident orstroke patients, extremely wealthy persons who can purchase these scarceservices at any price, and substantially random small groups who happento be “adopted” into some funded neuro-psychological research study.

Even within these groups, patients are not always able to receiveoptimal neurological care because of the time required for each test andits analysis, the time required to develop new tests, and the timerequired for the determination of the exact combination and sequence oftests to properly evaluate a given individual, which sequence may bedifferent at each examination of the individual. Neuro-psychologyprofessionals must spend a substantial amount of time administeringtests that often yield little or no useful data. They cannot spend thevast amount of time which would enable them to observe patients invarious circumstances, including routine normal activities, observationwhich would yield information enabling assessment of an individual'scondition at a given point in time, detecting the appearance of a newcondition, and perceiving the changes in and progress of a knowncondition.

Neuro-psychology professionals are furthermore faced with limitations indeveloping new tests to diagnose specific conditions more accurately orquickly, again, often because of time and cost limitations. Theneuro-psychology professional has limited means available to him tovalidate new tests, or even to correlate information gained from theperformance of routine activities by an individual under normalconditions or those experiencing changes in such conditions.

In cases when a neuro-psychology professional does develop and validatea new test, he is faced with a further problem; that of collectingpayment, or royalties, for the use of the test by others.

Given these pressing circumstances, there are clearly several distinctneeds in the present art. There is a need for ways and means ofsuccessfully training more persons in the complex applied-knowledgepractices of clinical neuro-psychology, leading to an increase in thenumber of properly certified neuro-psychology professionals.Independently, there is also a need for ways and means of successfullyextending the benefits of care, under the auspices of certifiedneuro-psychology professionals, to respectively larger populations ofpatients, who substantially need their help, albeit perhaps to morevarying degrees than members of the four select sub-populations that aredescribed above.

Neuro-psychology professionals undergo a long, complex and arduouscourse of study and apprenticeship. All too few succeed. This results ina very limited number of clients, of a potentially very large number ofclients, being able to benefit from these important professionals.Unfortunately, even this small number of clients only benefit to arelatively limited degree. One of the factors that exacerbates thislimit, is the availability of up-to-date and normal routine activitytests. Another factor is the limited range of conditions that will yieldto the relatively out-of-date testing methods available. For theneuro-psychological professional, the development and proper validationof normal activity tests, gives rise to the difficulty of financialrecovery of the costs involved as well as a return on the investment oftime and substantial effort.

There is, moreover, a need to extend the level of care that aneuro-psychgology professional is able to give specific groups ofpatients by increasing the efficiency of the testing and observationprocedures, thus enabling the observation of a patient's conditionthroughout the day or over an extended period of time, testing both byobservation of a patient's routine normal activities and by enablingmultiple or duplicate testing throughout the day. Furthermore, there isa need for a method that will enable greater efficiency in thevalidation of new tests and correlation of data from new tests withknown parameters. Finally, there is a need for a means to enableneuro-psychological professionals to collect royalties for tests, whichthey have developed and which are to be used by others.

There is a distinct need in the art to allow neuro-psychologistprofessionals to deal with a larger number of clients and to be able tomake better use of the time available. In addition, this need will besubstantially enhanced if better use can be made of existingstandardized tests. Add to these aspects, the development of and theability to validate new types of tests which can fulfill all these needswhilst providing a better professional-to-client relationship with theinevitable improvement in client evaluation and treatment.

Advantages, Objects and Benefits of the Invention

Technical Issues: The present invention delivers, to individual or smallgroups of certified neuro-psychology professionals, an interfaceprotocol for clinically monitoring and managing the neuro-psychologicalneeds of very large numbers of individual patients. The preferredapproach to such monitoring and testing, is within the scope of normaland routine activities. These patients are most often in need of regulartesting, screening, and monitoring and, only occasionally, of referralor intervention.

Ergonomic Issues: The present invention provides a majority of patients,with regular testing, screening, and monitoring services on a“virtual-out-patient” basis by interactively delivering individuallyconfigured batteries of tests to each patient, via the Internet. Thesetests will include normal activity observation and monitoring. Thedelivery and corresponding acceptance of the testing batteries is asubstantially automatically scheduled feature of the protocol of thepresent invention. Normal anxiety, stress and lost time of the patient,all of which are accepted and associated with prior methods of providingsuch testing in a clinical office setting, are all simultaneouslyremedied by the present invention. The present invention provides amultiplicity of services to patients in their respective daily settings,such as at home, at work, or even in transit using ordinary cellularwireless interface data-communications infrastructures as well asprovides a means of accumulating a data base of normal activity testingfor subsequent validation. Furthermore, certified neuro-psychologyprofessionals using the present invention are able to provide a higherstandard of ordinary care to many or even all residents of institutionalfacilities, such as hospitals, rehabilitation centers, and otherlife-care service-providing support-structures. Because of the presentinvention, neuro-psychology experts will become able to amass theinvaluable resource of a large scale database including a means ofvalidating normal activity testing, and thereby be able to describe newstatistically validated clinical longitudinal trends and other newlife-saving clinical correlations.

Economic Issues: Two fundamental economic improvements are achievableusing the protocol of the present invention. Because of the presentinvention, patients, who otherwise might have only benefited from theneuro-psychology experts' consultation if circumstances had becomelife-quality critical, may now find that these services are availableand affordable. Because of the present invention, neuro-psychologyexperts will be able to provide minimal care at low cost to a broadclass of less than severely disabled persons, who have not, heretofore,had reasonable access to such services. Furthermore, because of thepresent invention, neuro-psychology experts will be able to modestly andefficiently conduct countless low cost neuro-psychological researchstudies, most especially related to normal routine activities, sincesuch studies will become a virtual laboratory between the experts'suspicions and the reality of the data base.

Notices

Numbers, alphabetic characters, and roman symbols are designated in thefollowing sections for convenience of explanations only, and should byno means be regarded as imposing particular order on any method steps.Likewise, the present is herein described with a certain degree ofparticularity. However, those versed in the art will readily appreciatethat various modifications and alterations may be carried out withoutdeparting from either the spirit or scope of the present invention, ashereinafter claimed.

In describing the present invention, explanations are presented in lightof currently accepted scientific, technological or medical theories andmodels. Such theories and models are subject to changes, both adiabaticand radical. Often these changes occur because representations forfundamental component elements are innovated, because newtransformations between these elements are conceived, or because newinterpretations arise for these elements or for their transformations.Furthermore, the present invention will optimistically actuallycontribute to changing these theories and models. Therefore, it isimportant to note that the present invention relates to specifictechnological actualization in embodiments. Accordingly, theory or modeldependent explanations herein, related to these embodiments, arepresented for the purpose of teaching the current man of the art or thecurrent team of the art, how these embodiments may be substantiallyrealized in practice. Alternative or equivalent explanations for theseembodiments may neither deny nor alter their realization.

Glossary

Tests are computationally quantifiable instruments designed to assessone or more brain functions, including mental or motor functions.

A keyword that is used often in the occupational therapy world is“activities of daily living (ADLs)”. ADLs are substantially concernedwith the brain functions that contribute to a person's abilities to doall daily activities, including highly skilled activities. Theseactivities include ‘common tasks’ such as cutting food, writing,bathing, cooking, or ‘skilled’ vocational tasks such as performingsurgery, watch making, or serving as a corporate executive. These arethings that make up the person, in his interaction with the environment.Testing relates to accessing a metric related to any elemental brainfunction that contributes to these tasks. These metrics may not alwaysbe related to classical “clinical neuro psychology”.

Client or patient—person being tested.

Clinician or neuro-psychology professional—person prescribing orrecommending the tests, such as a medical professional, a clerk incharge of vocational testing, a school social worker involved ininvestigating poor scholastic performance, a motor vehicles departmentclerk, etc.

Normal activities testing relates to testing and monitoring, whilst aclient is carrying out tasks related generally to the use ofcommunication devices and related equipment.

SUMMARY OF THE INVENTION

The present invention relates to a software driven protocol for managinga virtual clinical neuro-psychological testing program, the protocolincluding for each client of a plurality of clients the steps:

a) evaluating a prior history of the client;

b) according to the evaluated prior history, forming an appropriatebattery of tests for testing the client wherein the battery incorporatespseudo-randomization of at least one representational or organizationalparameter; and

c) via a data-communications medium, interactively

i. delivering, to the client, the formed battery of tests; and

ii. accepting, from the client, a substantially completed response tothe delivered formed battery of tests;

d) analyzing the accepted response; and

e) into the prior history of the client, integrating

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

In general, this invention relates to software setting out a set ofrules whereby a clinician will be able to manage a computer operatedneuro-psychological series of tests for one or more clients. Thisprocess, with a large number of clients, is started with obtaininginformation from each client for the purpose of assessing testingrequirements. On the basis of this information, a fairly standard seriesof tests are typically prepared for each client which include variousnon-specific tests or parts of tests, randomly inserted amongst thenecessary tests. These tests are then conveyed to the client through themeans of a medium such as the Internet or other communicationsinter-connection. After the client has substantially completed thetests, these are returned to the clinician as the tests progress, to beexamined in the light of the information initially supplied by theclient, with the objective of establishing a measure whereby theclinician is able to determine the client's condition.

This procedure enables the neuro-psychologist to format tests and then,while monitoring each test and continuously having evaluated theresults, by means of any one of a number of communication devices, toset further tests to more closely determine the client's situationwithout having to expend time in personal attendance on the client.Randomly inserted and apparently irrelevant tests have two majorpurposes. Firstly, they represent a means of interrupting the monotonyof a series of similar tests and secondly, these tests will provide ameans for developing additional tests for assessment and validation sothat the scope of testing is constantly being expanded.

Advantages to such a system and improvements to existing procedures isexcitingly significant. Professional neuro-psychologists will be in aposition to monitor many more clients, more frequently and morethoroughly, whilst simultaneously making a major advance in expandingthe library of validated tests.

Furthermore, the present invention relates to a clinical protocol fornormal-use activities, the protocol including the steps of:

a) for substantially each client in an ensemble of clients, monitoringat least one metric of normal-use activities;

b) for substantially each metric of the at least one metric of normaluse activities, until a predetermined threshold of validation isachieved for an ensemble of clients, first, managing a virtual clinicalneuro-psychological testing program for substantially each client in theensemble; and correlating analytical metrics derived from theneuro-psychological testing program with the monitored metrics ofnormal-use activities, thereby validating at least one of the monitoredmetrics of normal-use activities as a neuro-psychological metric; and

c) for substantially each validated metric of the at least one metric ofnormal use activities, second, managing a virtual clinicalneuro-psychological testing program for at least one client wherein avalidated normal-use activity metric is used as a classical testinginstrument.

In general the present invention also provides a means for developingtesting procedures related to normal activities, generally with regardto each of a number of clients' interactions with a computer or othercommunication device. Appropriate software enables the clinician torecord and later compare measurements derived from these normalactivities. This software provides the means for validating this normalactivity testing. Such validated series of normal activity tests providethe means, then, to utilize these validated normal activity tests on aclient.

The procedure described, enables a clinician to develop new validatedforms of testing whilst conducting existing forms of tests without beingphysically present with the client. The usefulness of these newlyvalidated tests, involving normal activities, cannot be over-emphasizedin the light of the present lack of normal activity validated tests.Clearly the optimum and preferred method of testing would be directobservation, on a continuous basis, of normal activities by a client.Monitoring normal activities and evaluating the client on the basis ofvalidated normal activity tests as outlined in the protocol describedabove could supercede lengthy direct, one-on-one observation.

It should be understood that this aspect of the present invention,though quite similar to the previous aspect, has some intrinsicallysignificant differences. The utopian concept of professionalneuro-psychologists being able to spend almost limitless time observingclient normal activity behavior, comes a step closer with an increase inthe available arsenal of testing materials. Also, tests can be formedand applied that are specific in terms of each client's needs to asubstantially larger number of clients and developed as normal activitytests rather than presently available rigid standard test batteries.

Notwithstanding the above, the present invention also relates to asoftware driven protocol for managing a virtual clinicalneuro-psychological dynamic hierarchical testing program, the protocolincluding for each client of a plurality of clients the steps:

a) evaluating a prior history of the client;

b) according to the evaluated prior history,

i. interactively forming an appropriate battery of tests for testing theclient wherein the battery incorporates pseudo-randomization of at leastone representational or organizational parameter;

ii. via a data-communications medium, interactively delivering, to theclient, the formed battery of tests;

iii. accepting, from the client, a substantially completed response tothe delivered formed battery of tests;

iv. analyzing the accepted response; and

v. returning to step b)i until complete; and

c) into the prior history of the client, integrating

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

In general this procedure for testing differs from the initiallydescribed procedure. At present, the initial procedure utilizes aselection of a battery of tests transmitted to a client. After asubstantially completed battery of tests is returned for evaluation,analyzed results are integrated into the historical informationoriginally supplied by the client. This initially described procedurehas a significant drawback, in that the whole structure of testing isinflexible and totally rigid in that completed tests are evaluated onlysubsequent to the client returning these. Generally, no provision ispresently made for progressive monitoring of results, emanating fromongoing tests.

The procedure, described here, makes provision for avoiding thispitfall.

A battery of tests is selected by a neuro-psychologist as is the presentpractice. As the tests progress, from the continuous stream of results,the clinician is able to evaluate results as a continuum and canwithdraw the running test, even mid-test. Substitute alternative testsare then introduced that will give a better insight into the client'ssituation as well as the means for deriving a narrower distribution ofresults. This dynamic process of replacing ongoing tests, in ahierarchical way, may be repeated as often a necessary to achieve betterand more specific results. Again, a specific stress is laid on the useof collecting and collating the results of the normal activity tests,concurrent with the above process. These normal activity results arethen validated to provide normal activity testing facilities for use onclients.

Present conventions of neuro-psychological testing limit professionalsto a small number of clients, who can only be monitored for a verylimited amount of time, as opposed to the improvements that this dynamichierarchical protocol envisages. More appropriate use can be made of awider range of tests available from the usage of normal activity testingas well as the ability to monitor many more clients by means ofcommunication devices. Most significant is the avoidance of the rigidityof existing testing methods insofar as tests envisaged in these aspectsof the present invention, allow for a continuous interchange of resultsand tests between client and professional via the communication medium,enabling the professional dynamically to vary and to guide the testshierarchically, to obtain results that will be more specific to theclient's situation.

An additional aspect of the present invention relates to a softwaredriven protocol for managing a virtual clinical neuro-psychologicaltesting program based on normal use activities, the protocol includingfor each client of a plurality of clients the steps:

a) evaluating a prior history of the client;

b) according to the evaluated prior history, forming at least one normaluse activity factor of tests for testing the client wherein said atleast one normal use activity factor incorporates pseudo-randomizationof at least one representational or organizational parameter; and

c) via a data-communications medium, interactively

i. delivering the formed at least one normal use activity factor teststo the client;

ii. the client substantially completing the normal use activity factortests; and

iii. accepting, from the client, a substantially completed response tothe delivered normal use activity factor tests;

d) analyzing the accepted response; and

e) into the prior history of the client, integrating

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

In general, the invention also defines a procedure for carrying outtesting procedures elaborated in the earlier parts of this description,but in this instance, the tests are built up, in a progressive manner,using the concept of previously validated normal activities testing.

Again, the neuro-psychologist specifies a software driven battery oftests. Again, the inclusion of apparently non-specific testsinterspersed amongst the other tests is carried out to provide theclient with a rest from the rigors of the ongoing testing as well asproviding additional test information for later validation. Tests areconveyed to the clients through the means of a communications mediumusing a computer system or other personal communication device and,while being substantially completed, are returned to the clinician forrecording, collating, contrasting and inclusion with the earlierrecorded history of the client. It is stressed that this entireprocedure relates to validation of and subsequent use of normalactivities testing. This then is the optimal and preferred way oftesting clients, without the continuous participation of theprofessional neuro-psychologist in attendance with each separate clientand without the client being subjected to presently utilized lengthy,generally rigid and often monotonous tests.

In addition, the insertion of randomly placed and apparentlyinconsequential tests into each of the above test procedures, apart frombreaking the monotony of testing, provides a means for developing new,innovative and perhaps novel, forms of neuro-psychological testing.Clearly, validation of these newly developed tests is a naturalcorollary to all of the above.

The ideal way for professional neuro-psychologists to carry out testingis to expend large amounts of time monitoring each individual clientwithin the framework of normal and routine activities. This is notreally possible when considering the lack of sufficient qualifiedprofessional experts. There are many aspects to the present inventionthat will emulate this idyllic scenario. Not only will the scope of eachprofessional be expanded in terms of the number of clients who can besimultaneously monitored, but this monitoring will be in the context ofnormal activities testing. In addition, the scope of testing facilitieswill be expanded on an ongoing basis, as a consequence of the use ofpseudo randomly incorporating normal activity tests.

The major thrust of this invention, is the development of a broadspectrum of innovative test methods, which will provide the professionalneuro-psychologist with the means of dealing with substantial numbers ofclients without the need to expend long hours of direct contact withclients. Professionals will also be giving a much higher quality ofservice to these clients. Most significant is the use of normalactivities testing through the medium of commonly used communicationdevices.

To summarize, there is a specific problem that presently exists. Expert,professional neuro psychologists are unable to cope with the existingwork load of clients requiring direct, continuous and long termmonitoring in terms of their normal routine activities, especially usingthe generally rigid types of testing batteries available. The presentinvention provides a means of overcoming all these problems. Clients canbe monitored directly on a continuous and interactive basis viacommunication media during performance of their normal routineactivities, with the professional having the means to vary testingprocedures as the pattern of client tests indicates a direction forachieving more direct and narrower results, specific to each client'sneeds. Finally, the professional will be able to so monitorsubstantially larger numbers of clients, using these much improved andvalidated test methods. Validation of testing methods will naturallyfollow this new scenario, producing a much broader and improved scopefor the professional neuro psychologist.

BRIEF DESCRIPTION OF THE FIGURES

In order to understand the invention and to see how it may be carriedout in practice, embodiments including the preferred embodiment will nowbe described, by way of non-limiting example only, with reference to theaccompanying drawings, in which:

FIG. 1 illustrates a schematic view characterizing the steps of asoftware-driven protocol for managing a virtual clinicalneuro-psychological testing program;

FIG. 2 illustrates a schematic view characterizing the steps of aclinical protocol for normal use activities;

FIG. 3 illustrates a schematic view characterizing the stepscorresponding to first managing and second managing in the clinicalprotocol for normal use activities;

FIG. 4 illustrates a schematic view characterizing the stepscorresponding to first managing and second managing in the clinicalprotocol for validating a new testing instrument;

FIG. 5 illustrates a schematic view characterizing the steps of asoftware driven protocol for managing a virtual clinicalneuro-psychological dynamic hierarchical testing program;

FIG. 6 illustrates a schematic view characterizing the steps of theprotocol for validating a non-validated test in a formed battery ofvalidated tests by correlating responses from the validated andnon-validated tests;

FIG. 7 illustrates a schematic view characterizing the steps ofestablishing, upgrading and maintaining in the protocol for validating anon-validated test in a formed battery of validated tests by correlatingresponses from the validated and non-validated tests;

FIG. 8 illustrates a schematic view characterizing the steps of asoftware driven protocol for managing a virtual clinicalneuro-psychological testing program based on normal use activities;

FIG. 9 illustrates a schematic view characterizing the steps of firstmanaging and second managing in the protocol for validating anon-validated test in a formed battery of validated tests by correlatingresponses from the validated and non-validated tests;

FIG. 10 illustrates a schematic view characterizing the steps of acomputer program product including a computer usable medium havingcomputer readable code embodied therein for a software driven protocolfor managing a virtual clinical neuro-psychological testing program;

FIG. 11 illustrates a schematic view characterizing a program storagedevice readable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for a software drivenprotocol for managing a software driven protocol for managing a virtualclinical neuro-psychological testing program;

FIG. 12 illustrates a schematic view characterizing a system overview ofa software driven protocol for managing a virtual clinicalneuro-psychological testing program;

FIG. 13 illustrates a schematic view characterizing the steps of thechoice of a test battery for pre-testing in accordance with the softwaredriven protocol for managing a virtual clinical neuro-psychologicaltesting program;

FIG. 14 illustrates a schematic view characterizing the steps of dataacquisition in accordance with the software driven protocol for managinga virtual clinical neuro-psychological testing program;

FIG. 15 illustrates a schematic view characterizing the steps of a TestBattery Wizard for use in discerning the nature of a battery of testsrequired with regard to a client's needs;

FIG. 16 illustrates a schematic view characterizing the steps of dataanalysis for each test carried out in accordance with the softwaredriven protocol for managing a virtual clinical neuro-psychologicaltesting program; and

FIG. 17 illustrates a schematic view characterizing the steps of theinterpretation of a test battery carried out in accordance with thesoftware driven protocol for managing a virtual clinicalneuro-psychological testing program.

DETAILED DESCRIPTION OF THE INVENTION

There are several aspects to the present invention. Using the presentinvention, the professional neuro-psychologist will be able to manage alarge number of clients on a professional basis and to utilize a systemand a method to influence creation of batteries of tests and evaluationparameters. This will provide the means to evaluate a client usingup-to-date instruments. Monitoring clients or a specific condition in aclient, through a battery of tests, will be done using a procedure,which will enable direct and ongoing contact between client andclinician, as necessary.

A substantial portion of the tests can be administered, evaluated, andmonitored automatically. Ongoing results fed back on an on-line basiswill be evaluated to determine the need for further testing and thenature of such testing. This automated process further provides theneuro-psychology professional with new parameters and evaluative metricsfor determining and assessing a neuro-psychological condition in aclient.

Furthermore, maintaining and improving a professionalneuro-psychologist's ability to manage a large client base with anappropriate level of professional service, is an essential factor ofthis invention. Each client requiring testing, will have the benefits ofpaying for actual services received, having access to a multiplicity oftests, have access to all appropriate professionals and a reduced burdenof testing.

However, it must be stated, that it is a specific desire of theneuro-psychology professional, to subject clients to as little testingas possible whilst obtaining a satisfactory diagnosis or reaching anaccurate conclusion. The interests of the client are best served bybeing subjected to a minimum burden of stress, trauma, anxiety or simplebother, during the testing procedure.

Reference is now made to FIG. 1, which illustrates a schematic viewcharacterizing the steps of a software-driven protocol (100) formanaging a virtual clinical neuro-psychological testing program.

The present invention relates to a software-driven protocol for managinga virtual clinical neuro-psychological testing program, the protocolincluding for each client of a plurality of clients the steps:

a) evaluating (102) a prior history (101) of the client;

b) according to the evaluated prior history, forming (105) anappropriate battery of tests (106) for testing the client wherein saidbattery incorporates pseudo-randomization (107) of at least onerepresentational or organizational parameter; and

c) via a data-communications medium (110), interactively

i. delivering (109), to the client (111), the formed battery of tests,and

ii. accepting (112), from the client, a substantially completed responseto the delivered formed battery of tests;

d) analyzing (113) the accepted response; and

e) into the prior history of the client, integrating (114)

i. the accepted substantially completed response, or

ii. at least one analytical metric thereof.

Each of a multiplicity of clients will be provided with specificsoftware, which will enable a professional clinician to maintain aninterchange of information using corresponding software. This will allowa client to feed historical information to the clinician, receivetesting procedures and instructions, carry out tests and convey resultsto the clinician for evaluation and integration into a client's historyin order to determine further testing requirements or to drawconclusions regarding each client's situation.

Simply stated, evaluating a prior history of the client is a clinician'splacing an emphasis on automatic determination of the tests needed basedon the history of a client or, alternatively, a clinician shifting theemphasis to a higher level of involvement in the progress of the client.

More specifically, forming an appropriate battery of tests, is theconsequence of a clinician's evaluation of the condition of a client.Such a battery of tests will comprise a variety of tests includingarranging some of the above-mentioned representational or organizationalparameters in a random manner. The reason for applying a randomrearrangement, revolves around the problem that clients tend to learntests and, consequently, these no longer measure what was intended orrequired. To avoid this problem it is important that tests are notrepeated at respective known close intervals. Simply stated, re-use ofthe same testing instrument is generally restricted according to theprotocol that validated that instrument. Nonetheless, randomization maybe accomplished by changing the order of elements in a test or bychanging around actual examples in any question. Introducing thesedifferent levels of changes ensures maintaining the efficacy of usingrepeated testing instruments at appropriate intervals.

Initially, the tests that are formed into a battery, will be appropriateto the client's evaluated situation. Using a data communications medium(110) interactively, implies sending test materials to a client andreceiving the substantially complete results from the client. The cycleof sending and receiving may occur as a single or as a series ofinterchanges, when administering a single battery. It is preferred thatthe remainder of the formed battery is modified after each interchangecycle; as will be described in detail below in the section describingdynamic hierarchical testing.

Morespecifically, delivering to the client, the formed battery of tests,takes the form of conveying testing instruments by means, for example,of the Internet, by email or other data communication media. Similarly,accepting from the client, will also take place using one of similardata communication media.

Analyzing the accepted response from the client, more specifically, isthe means by which a clinician will decide on comparing accepted resultsto normal or anticipated standards and then to decide whether to call ina physician or to deliver a further battery of tests. Further, byintegrating these analyzed results into the prior history of thepatient, the clinician becomes able to more easily determine the routeto be followed, in particular, with regard to further testing, ifneeded. Subsequently scheduled tests, more specific to a client'scondition or need, will be applied, following on a defining of thecurrent forming of a battery of tests to the next forming event and soto evaluate results.

To conclude, the cycle of events defined by the present invention,enables the fulfillment of the broad goals of the present invention. Aprofessional can deal with a large client population and the client willbe minimally subjected to invasive interaction with the professional.

More sophisticated varieties of the present invention, described below,will more efficiently make use of the professional's time and moreefficiently reduce the client's burden, without reducing the efficiencyof interaction between professional and client.

When performed by an expert, a standard neuro-psychological evaluationinvolves using a loosely structured battery of substantiallystandardized tests, that are progressively individualized to specificdiagnostic issues of each particular patient. Hence, the expert formsand then uses a custom configured battery of tests that provide athorough investigation of a patient, depending on the circumstances ofthe problem and the state of the patient.

Sometimes, a rapid screening examination may be adequate if the problemis minor or intermittent. On the other hand, screening may be all thatis possible if the condition or participation of the patient does notallow more detailed testing. If certain domains of function aresuspected to be abnormal, based on historical information or on positivefindings during screening tests, then more detailed testing of thosedomains is warranted.

It is apparent that the neuro-psychological professional will be able tomanage a larger number of clients using tools supplied by a broad rangeof experts. Simultaneously, billing and time management will besuperior. On the other hand, clients will receive a more cost-effectiveand professional service, and have the benefit of a significantlyreduced testing burden.

According to an embodiment of the present invention, the protocol inwhich analyzing the accepted response includes: applying (119)rule-based criteria for quantifying (120) at least one differencebetween a pair of metrics selected from the list:

a) the evaluated prior history (121) of the client;

b) a metric of expected performance (122) on at least one test of theformed battery of tests;

c) actual performance (123) on at least one test of the formed batteryof tests; and

d) normative values (124) based on large body of data for at least onetest of the formed battery of tests.

More specifically, results will be evaluated and applied to a selectionof additional tests, if required, by comparing two of the metrics,evaluated prior history, a measure of expected performance, actualperformance and normal performance from a body of previously developeddata. This enables a system embodying the present method to determinewhether the next level of interaction of tests is warranted.

A comparison of the evaluated prior history of a client with a metric ofexpected performance will indicate whether the selected battery of testsis likely to be appropriate. Comparing the evaluated prior history witha metric of actual performance will enable the clinician to determinethe subsequent course of action, namely to apply further, more specifictesting instruments, to call in the services of a physician or to decidethat no further testing is indicated at the present time. Comparing theevaluated history with normative test values, based on a large body ofdata, enables the clinician to anticipate testing results so that he canpreempt the need for the client to carry the testing instrument to itsconclusion or to replace the test with another, more appropriate or todecide that no further testing is indicated at the present time.

Further, to compare a metric of expected performance with actualperformance, confirms, for the clinician, the correct direction of, andthe appropriateness of the selected testing instruments. Also, comparingthe expected performance with normative values for a metric of the sametest enables the clinician to determine the nature of the condition ofthe client, relative to the norm, in the light of the evaluated historyof the client. Finally, comparing actual performance against normativevalues, gives the clinician an insight into the nature of the conditionof the client and to decide whether further specific testing is calledfor or whether consultation with other expert professionals isindicated.

Reference is now made to FIG. 2 which illustrates a schematic viewcharacterizing the steps of a clinical protocol (200) for normal useactivities, the present invention also relates to a clinical protocolfor normal-use activities, the protocol including the steps of:

a) for substantially each client in an ensemble of clients (211, 212,213, etc.) monitoring (201) at least one metric of normal-useactivities;

b) for substantially each metric of the at least one metric of normaluse activities, until a predetermined threshold of validation isachieved for an ensemble of clients, first managing (202) a virtualclinical neuro-psychological testing program (205, 207, 209, etc.) forsubstantially each client in the ensemble, and correlating (203)analytical metrics derived from the neuro-psychological testing programwith the monitored metrics of normal-use activities, thereby validatingat least one of the monitored metrics of normal-use activities as aneuro-psychological metric; and

c) for substantially each validated metric of the at least one metric ofnormal use activities, second managing (204) a virtual clinicalneuro-psychological testing program for at least one client (214)wherein a validated normal-use activity metric is used as a classicaltesting instrument.

More specifically, second managing applies to using the technique ofusing validated normal activities testing. In particular, what has beenlearned about normal activities, contributes to the way clients' testingprograms are managed.

Current practice requires the clinician to administer aneuro-psychological test, substantially, in the form in which the testwas validated. Generally, this involves many repetitions over extensivetesting sessions. There are currently no validated automated models formaking decisions regarding test termination or switching to a differenttest during the course of the testing session. To partially alleviatethis burden on the client, the concept of normal activities testing isintroduced.

More specifically, software will provide a means for monitoring a largenumber of clients during respective normal activities (206, 208, 210,etc.) managing each client until a sufficient level of validation isachieved by correlating results, and then allowing evaluation ofclients' normal activity use with the validated test.

Of necessity, the optimal method of monitoring such normal activitiestesting, is supplemented with direct on-line testing. This would seem torequire a reduced level of apparent participation by a client and,therefore, substantially less trauma and stress for a client.Simultaneously, the clinician is able to deal with a plurality ofclients improving the time efficiency and hence the cost effectivenessof such testing.

FIG. 3 illustrates a schematic view characterizing the stepscorresponding to first managing and second managing (300) in theclinical protocol for normal use activities. In addition, FIG. 9illustrates a schematic view characterizing the steps of first managingand second managing (900) in the protocol for validating a non-validatedtest in a formed battery of validated tests by correlating responsesfrom the validated and non-validated tests.

According to a further embodiment of the present invention, the clinicalprotocol for normal-use activities in which occur first managing andsecond managing, include the steps:

a) evaluating (305) or (902), a prior history (302) or (901) of theclient;

b) according to the evaluated prior history, forming (306) or (903) anappropriate battery of tests (307) or (904) for testing the clientwherein said battery incorporates pseudo-randomization (308) or (905) ofat least one representational or organizational parameter; and

c) via a data-communications medium (310) or (907), interactively

i. delivering (309) or (906), to the client (311) or (908), the formedbattery of tests; and

ii. accepting (312) or (909), from the client, a substantially completedresponse to the delivered formed battery of tests;

d) analyzing (313) or (910) the accepted response; and

e) into the prior history (302) or (912) of the client, integrating(314) or (911)

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

According to a subsequent embodiment of the present invention, theclinical protocol for a metric of normal use activity of the monitoredat least one metric of normal use activities, is calculated by measuringan activity selected from the list of copying, printing, pasting,editing, inserting, formatting, exchanging tasks, web surfing,backspacing, deleting and shifting, substantially, as described earlierI DON'T SEE ANY OF THIS REFERENCED EARLIER in the examples of making useof personal computers, palm-held computers, cellular telephones, etc.

It is reasonable to presume, once systems are operated according to thepresent invention, other normal activity monitoring devices will becomestandard items requested by clinicians for the routine administration oftheir clients. These could include devices such as a telemetricneurology monitor, a telemetric heartbeat monitor, etc.

Reference is now made to FIG. 4, this illustrates a schematic viewcharacterizing the steps corresponding to first managing and secondmanaging (400) in the clinical protocol for validating a new testinginstrument, according to an additional embodiment of the presentinvention,

The clinical protocol for validating a new testing instrument, thatdescribes first managing and second managing include the steps:

a) evaluating (402) a prior history (401) of the client;

b) according to the evaluated prior history, forming (403) anappropriate battery of tests (404) for testing the client wherein saidbattery incorporates pseudo-randomization (405) of at least onerepresentational or organizational parameter; and

c) via a data-communications medium (407), interactively

i. delivering (406) to the client (408), the formed battery of tests,and

ii. accepting (409), from the client, a substantially completed responseto the delivered formed battery of tests;

d) analyzing (410) the accepted response; and

e) into the prior history (412) of the client, integrating (411)

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

There is apparent repetition of aspects and certain embodiments of thepresent invention but this is necessitated due to differences inemphasis. In the particular case, regarding validating a new testinginstrument that has been dealt with, the emphasis has changed frommanaging a testing program of validated testing instruments to usingnormal activities as part of the measure and validation of this form oftesting. Particularly, managing a testing program containing validatedtesting instruments, and using normal activities as part of thevalidation of normal use activities, will replace some of the knownvalidated testing instruments.

Validation of neuro-psychological tests is presently a cumbersome andextremely costly procedure. Clinicians are required to administer testsin the form in which these were validated. Therefore, development ofnormal activity testing procedures as well as the introduction ofvalidation techniques for these procedures, will enable a clinician tofulfill this criterion at minimal cost and with a minimum of directeffort.

More specifically, the effects of the utilization of such a testingprocedure on the client are manifold. Inevitably, present testingprocedures cause stress and trauma and, in many instances, render littleor no useful results, whereas normal activities testing should bevirtually free of such problems. With the present invention, the clientwill have a benefit of expertise from a multitude of professionals inmany connected fields. Another significant factor will certainly relateto substantially reduced costs.

FIG. 5 illustrates a schematic view characterizing the steps of asoftware driven protocol (500) for managing a virtual clinicalneuro-psychological dynamic hierarchical testing program.

The present invention further relates to a software-driven protocol formanaging a virtual clinical neuro-psychological dynamic hierarchicaltesting program, the protocol including for each client of a pluralityof clients the steps:

a) evaluating (504) a prior history (501) of the client;

b) according to the evaluated prior history,

i. interactively forming (505) an appropriate battery of tests (506) fortesting the client wherein said battery incorporatespseudo-randomization (507) of at least one representational ororganizational parameter;

ii. via a data-communications medium (511), interactively (512)delivering (510), to the client (513), the formed battery of tests,

iii. accepting (514), from the client, a substantially completedresponse to the delivered formed battery of tests;

iv. analyzing (515) the accepted response; and

v. returning (508) to step b)i until complete; and

c) into the prior history (518) of the client, integrating (517)

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

Simply stated, evaluating a prior history of the client, includesdetermining the basis on which a battery of tests will be constructed totest specific aspects regarding the client. Further, forming thisappropriate battery of tests incorporating pseudo-randomization, isspecifically for the purpose of ensuring that repeating these tests at alater time, will prevent the client from learning to perform the testsand thus, defeat the objective of the tests. In addition, deliveringthese formed batteries of tests and later accepting the substantiallycompleted tests, using a data communication medium, facilitatesaccomplishing the purpose of interactively analyzing the test resultsand of delivering additional or more specific tests. This cycle ofevaluating, forming tests, delivering and accepting these tests, andanalyzing the results, is repeated until the clinician is satisfied withthe results. Integrating these into the prior history of the client, todetermine the course to be followed with respect to the client, willassist in deciding whether further testing or whether consultation withother experts is needed.

Once again apparent repetition of aspects and certain embodiments of thepresent invention is necessitated due to differences in emphasis. In theparticular case that has been dealt with, relating to a software-drivenprotocol for managing a virtual clinical neuro-psychological dynamichierarchical testing program, the emphasis has changed from managing atesting program and from validation of normal activities testing, tomanaging a virtual clinical neuro-psychological dynamic hierarchicaltesting program.

More specifically, managing a virtual clinical neuro-psychologicaldynamic hierarchical testing program, implies allowing testing programsto proceed only to a point at which the result will permit a better ormore specific test or battery of tests to be applied. This will occurwithout the client having to, necessarily, complete the originallyformed battery of tests.

There are currently no validated automated models for making decisionsregarding test termination or test switching during the course of atesting session. Embodied in the present invention is a means for usinga model whereby performance of neuro-psychological tests is evaluatedon-line. An automated decision process is used to determine one ofseveral options, including continuation of the test, prematuretermination of the test, change in level of difficulty of the same test,or switching to another test.

For example, for each level of difficulty for each test, an upper andlower boundary form the range of sensitivity for performance at thatlevel. Performance may be determined abnormal for that level based onpredetermined values of certain performance parameters in relation tosaid boundaries. Decisions to terminate the test on the basis of poorvalidity may be based on indices indicative of poor cooperation with thetest instructions or of fictitious results from malingering (using knowntechniques to detect inconsistencies in the results). Tests may beterminated based on general poor performance signifying severe generalcognitive dysfunction, or poor performance in specific domainssignifying focal deficits worthy of more focused testing.

Within a given test, a hierarchical design is used to move the testthrough different levels of difficulty. This is accomplished, fordifferent types of tests, by utilizing the principles of utilizing aseries of tests having changing levels of difficulty. These are:

a) Screening for subtle deficits may be accomplished by starting at thelevel of difficulty expected for normal individuals. If performance isin the range of normal, then the test is terminated. If there isabnormal performance, the test is repeated at a lower level ofdifficulty, until satisfactory performance is accomplished. For example,the common eye chart exam for visual acuity may be administered as ascreening test by starting with the smallest letters. If the subjectreads the lowest line satisfactorily, the exam is terminated. If not,the subject is presented with a line of larger letters until he is ableto read the letters satisfactorily.

b) When seeking the highest level of performance, such as in vocationaltesting, the level of difficulty may start with that of averageperformance and may gradually increase until performance becomesinadequate or abnormal.

c) When quantification of the performance is required, such as inpatient longitudinal follow-up during a drug rehabilitation ortherapeutic program, the level of difficulty may start at the subject'sprevious level of performance and either increase or decrease dependingon the performance.

Decisions to switch to other tests within the prescribed battery or totests outside of the original battery may be made based on testperformance measured on-line:

a) If enough information has been garnered from a given test todetermine whether performance is normal or, alternatively, to determinethe degree of abnormality, a decision may be made to move to the nexttest in the given pre-determined battery.

b) If a specific domain of abnormality has been identified, a decisionmay be made to switch to one or more test that further expand upon thearea(s) of abnormality.

FIG. 6 illustrates a schematic view characterizing the steps of theprotocol (600) for validating a non-validated test in a formed batteryof validated tests by correlating responses from the validated andnon-validated tests.

According to another embodiment of the present invention, the protocol(600) further includes the steps of:

a) Establishing (601), upgrading (602) and maintaining (603) asappropriate, a validity metric (611) for a substantially non-validatedtest (604), for each client (607) of an ensemble of clients;

b) introducing (605) a substantially non-validated test into the formedbattery of tests (606);

c) analyzing (608) responses to the substantially non-validated tests;and

d) correlating (610) responses with the responses from the other tests(609) in the formed battery of tests.

Existing data, describing the significance of normal use activities, is,in the main, weak. The simplest advantageous use of normal use activitymonitoring is to create advisory notices to the attending professionalwhenever normal use changes occur. The professional, then, can decide tobring forward further testing.

Alternatively, in the event that there are no changes in monitorednormal use, an attending professional, in many circumstances, is likelyto approve an occasional postponement of a testing event. Again, this isbased on his professional assessment of the client and supplemented byan observation of stability or improvement in a normal use metric.

The introduction of monitoring normal use activities for advancing orpostponing testing events will, inevitably, result in accumulation of alarge ensemble of data. From this data, monitored normal use can becorrelated with validated testing instruments. It is not unlikely topresume that metrics of monitored normal use activities will becomevalid substitutes for numerous clinical diagnostic testing instruments.

This also supports the objective of the present invention, allowing theprofessional to manage a large body of clients and reduce the testingburden of clients.

More specifically, results of applied non-validated normal activitiestests are compared with results of validated tests. This techniqueallows for the expansion and validation of normal activities tests.Resulting from this, it becomes possible to use normal activities forproviding valid but non-burdensome tests, especially, but notexclusively, for clients where more conventional testing is problematic.Normal activities testing methods become, then, a classical testinginstrument.

FIG. 7 illustrates a schematic view (700) characterizing the stepsindicated in FIG. 6 of establishing (601), upgrading (602) andmaintaining (603) in the protocol for validating a non-validated test ina formed battery of validated tests, by correlating responses from thevalidated and non-validated tests.

According to a variation of embodiments of the present invention, theclinical protocol for normal-use activities, establishing, upgrading andmaintaining, include:

a) for substantially each client (701) in an ensemble of clients,monitoring (703) at least one metric of normal-use (702) activities;

b) for substantially each metric of the at least one metric of normaluse activities, until a predetermined threshold of validation isachieved for an ensemble of clients, first managing (704) a virtualclinical neuro-psychological testing program for substantially eachclient in the ensemble; and correlating (706) analytical metrics derivedfrom the neuro-psychological testing program with the monitored metricsof normal-use (705) activities, thereby validating (707) at least one ofthe monitored metrics of normal-use activities as a neuro-psychologicalmetric; and

c) for substantially each validated metric of the at least one metric ofnormal use activities, second managing (708) a virtual clinicalneuro-psychological testing program for at least one client (709)wherein a validated normal-use activity metric (710) is used as aclassical testing instrument.

The consequences of using this software are significant for each of theparticipants. Benefits to neuro-psychologist professionals will includebeing able to easily bill for services, the ability to better control amuch larger client population, and availability of significantly moreup-to-date testing materials. Furthermore, it will provide a facileconduit to consult with appropriate specialists, easily enable theprofessional to manage referrals and to compare his evaluation with anexpert system.

Clients, however, will benefit by making payment only for servicesactually utilized or needed, results of testing will be easilyevaluated, a much broader range of testing materials will be available,and access to a wider range of professional services will be feasible.Significantly, the burden in terms of stress and trauma will be muchreduced.

For the researcher, there are also significant advantages. These are:access to a large, well managed client population; an environment tointroduce weakly validated testing instruments and improve theirvalidation; well-structured pool of background data to correlate resultsfrom testing instruments—allowing validation of these instruments at alower cost; and provide an easy way to market these now validated testsand receive remuneration, for these now validated tests.

Turning to FIG. 8 which illustrates a schematic view characterizing thesteps of a software driven protocol (800) for managing a virtualclinical neuro-psychological testing program based on normal useactivities

The present invention furthermore relates to a software-driven protocolfor managing a virtual clinical neuro-psychological testing programbased on normal use activities, the protocol including for each client(801) of a plurality of clients the steps:

a) evaluating (802) of a prior history (804) of the client;

b) according to the evaluated prior history, forming (805) at least onenormal use activity factor of tests (806) for testing the client whereinsaid at least one normal use activity factor incorporatespseudo-randomization (807) of at least one representational ororganizational parameter;

c) via a data-communications medium (809), interactively (810)

i. delivering (808) the formed at least one normal use activity factortests to the client (811),

ii. the client substantially completing the normal use activity factortests, and

iii. accepting (812), from the client, a substantially completedresponse to the delivered normal use activity factor tests;

d) analyzing (813) the accepted response; and

e) into the prior history (801) of the client, integrating (814)

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

This further apparent repetition of aspects and certain embodiments ofthe present invention has been necessitated due to additionaldifferences in emphasis. In the particular case that has been dealtwith, the emphasis has changed from managing a testing program, fromvalidation of normal activities testing and from managing a virtualclinical neuro-psychological dynamic hierarchical testing program, tomanaging a virtual clinical neuro-psychological testing program based onnormal use activities.

More specifically, there is a presumption that there are validatedtesting instruments. Nevertheless, substantially everyneuro-psychologist of an expert level agrees that there is nosubstantial substitute for observing the client and interacting with theclient until such time as the attending professional is convinced thathe has an understanding of inter-related functional aspects of theclient's state.

According to any of the embodiments of the present invention, theprotocol in which forming an appropriate battery of tests includeschoosing in FIG. 1 (106), in FIG. 3 (307), in FIG. 4 (404), in FIG. 5(506), and in FIG. 8 (806), at least two tests for measuring at leastone mental or motor function, and the tests are selected from the listof testing instruments for measuring: mental health, intelligence, IQ,memory, immediate recall, memory encoding, memory retrieval, workingmemory, semantic memory, procedural learning, sequence learning,conditioned response, Pavlovian learning, associative learning, implicitlearning, explicit learning, block learning, motor learning, patternmatching, judgment, attention, concentration, visual-spatial perception,velocity perception, distance perception, visual searching,calculational ability, mathematical ability, abstract thinking, symbolicthinking, adaptation, sensory-motor adaptation, language, reading,naming, comprehension, classification, direction-following, vigilance,motor, sensory-motor, coordination, psychomotor performance, dexterity,motor skills, tremor, physiological tremor, simple reaction time, choicereaction time, sustained attention, selected attention, dividedattention, driving safety, ballistic movement, bradykinesia,hypokinesia, akinesia, hypometria, movement speed, movement smoothness,movement accuracy, repetitive movement, accurately timed movements,bimanual coordination, hand-eye coordination, personality, scholasticperformance, depression, psychosis, neurosis, anxiety, stress,post-traumatic stress, dementia, static visual acuity, dynamic visualacuity, handwriting analysis, speech analysis, voice tremor, or metricsof interacting with a testing instrument.

According to any of the embodiments of the present invention, theprotocol in which forming occurs, includes for at least one test in thebattery of tests defining a subset of equivalent validation of testingobjects and therein pseudo randomization includes randomizing amongstthe substantially validated testing objects.

Simply stated, an additional means of validation of testing may befacilitated in accordance with embodiments to the present invention. Byinsertion of a number of condition non-specific tests randomly withinestablished and validated tests and batteries of tests, the clinicianwill be able to accumulate data to validate these inserted tests orsub-tests.

Expanding on the scope and multiplicity of neuro-psychological testingwill enable the professional clinician to expand the scope and depth ofinvestigative testing and will improve the ability to manage asubstantially larger client base with improved levels of professionalservice.

According to any of the embodiments of the present invention, theprotocol in which evaluating occurs, includes a disclosure ofinformation by the client.

According to any of the embodiments of the present invention theprotocol in which evaluating of a prior history of the client occurs,includes: interactively merging (104), (503) or (804), practitionerrecommendations (103), (502) or (803), for this client into the priorhistory.

According to any of the embodiments of the present invention, theprotocol further includes the step of scheduling a next evaluating of aprior history of the client.

Notwithstanding the existence or the original or current mode ofarranging monitoring of the client, scheduling a next evaluating of aprior history of the client means simply scheduling the next testingselection, or alternatively, scheduling describes bringing schedulingout of automatic mode to the eyes of the expert professional. Even ifnormal use activity has no validation as a metric, nevertheless,monitored normal use should either cause the next scheduling to beadvanced or should cause the professional to re-certify his evaluation.

According to any of the embodiments of the present invention, theprotocol in which analyzing the accepted response occurs, includes:calculating (117), an analytical metric of client performance using theaccepted response in the formed battery of tests of at least one clientperformance parameter from the accepted response of a test in the formedbattery of tests. This analyzing and calculating, simply stated, meansappreciating the significance of the results of a single variabletesting instrument

According to any of the embodiments of the present invention, theprotocol in which analyzing the accepted response occurs, includes:calculating (118), an analytical metric of client performance using theaccepted response in the formed battery of tests of at least oneconvergence parameter from the accepted response of at least two testsin the formed battery of tests.

In this calculating an analytical metric of client performance,neuro-psychological testing makes use of multi-parametric instead ofsingle variable factors. This analyzes and calculates the results ofmore than one variable converging to a significant definition of theclient's condition.

According to any of the embodiments of the present invention, theprotocol of a test of the delivered and accepted formed battery of testsincludes a metric of the clients interaction with a client's peripheraldevice, and the device is selected from the list: Color graphic display;B/W graphic display; Audio speaker; Audio stereophonic headphones;Mouse; Joystick; Roller-ball; Keyboard; Galvanic skin response monitor;Web-cam camera; Microphone; personal communication device; Touch pad,Touch screen; etc.

Simply stated, the client's testing will include interaction, from testto test, with a peripheral device, and includes examining such effectsas speed of operation, consistency of color discernment, mouse orjoystick tremor, speech variations, etc.

According to another embodiment of the present invention, the clinicalprotocol for substantially each client in an ensemble of clients,monitoring at least one metric of normal-use activities includesdownloading a plug-in (205, 207, 209, etc.) and (805) to the client'smachine, installing the plug-in, and uploading data collected by theplug-in, etc.

According to an embodiment of the present invention, in the clinicalprotocol for substantially each client in an ensemble of clients,monitoring at least one metric of device-dependent normal-useactivities, includes copying, pasting, editing, inserting, formatting,exchanging tasks, web surfing, keyboard backspacing, deleting andshifting, etc.

More specifically, the application of device dependent testing capableof software enablement as well as various data communication systems isimportant in the application of normal activities testing techniques,for example:

a) A personal computer, Internet enabled, where measurements arepossible, relating to keyboard use, mouse use, etc.

b) Palm-held computer which, on occasion, interacts with various typesof data communication devices, can be applied to normal activity testingusing the key-pad, using the search and select functions, using stylusand other data entering modes, etc.

c) Personal communication devices such as cellular telephones are alsoapplicable.

Making use of functions including key-pad use, device function uses likedialing or searching, voice print access interpreting the level ofvarious actions like slurring or decipherability, etc. are significantmeasures.

According to a subsequent embodiment of the present invention, in theclinical protocol for substantially each client in an ensemble ofclients, monitoring at least one metric of normal-use activities,includes computer related interactivity, wireless activity, voice printsfrequency analysis, mouse tremor, Geographic Positioning System tremor,key strokes, and special case use, etc.

Simply put, monitoring normal use activities such as those listed,enables the neuro-psychological professional to make decisions regardingthe state of a client, whether such testing has been validated or not.Significance can even be placed on monitoring unvalidated tests simplyby making comparisons over a period of time. Properly validated testswill give considerably more information to a clinician. Changes in amonitored factor would indicate to the professional, that a clientrequires further testing immediately or at some later time. Monitoringof these factors can be on a continuous basis or simply whenever theclient makes use of any of the data communication devices available.

A direct consequence to use of this software and plug-ins is:

a) maintaining and improving professional neuro psychologists' abilityto manage large client bases with an appropriate level of professionalservice,

b) billing control for testing becomes a straightforward matter for bothclient and clinician,

c) cost of testing will be substantially reduced,

d) time demands and requirements will be substantially reduced

e) providing clients with equal or lower test participation and simpleractive cooperation with test instruments,

f) the possibility of constant online contact and interaction betweenclient and clinician, and

g) reduced testing burden for the client.

FIG. 10 illustrates a schematic view characterizing the steps of acomputer program product (1000) including a computer usable mediumhaving computer readable code (1011) embodied therein for a softwaredriven protocol for managing a virtual clinical neuro psychologicaltesting program.

The present invention additionally relates to a computer program product(1000) including a computer usable medium having computer readableprogram code embodied therein for a software-driven protocol (1001) formanaging a virtual clinical neuro-psychological testing program, on theclinician side (1009), the computer readable program code in saidarticle of manufacture including: for each client of a plurality ofclients:

a) first computer readable program code for causing a computer toevaluate a prior history of the client;

b) tied to the first computer readable code, second computer readableprogram code for causing the computer, according to the evaluated priorhistory, to form an appropriate battery of tests for testing the clientwherein said battery incorporates pseudo-randomization of at least onerepresentational or organizational parameter;

c) tied to second computer readable code, third computer readableprogram code for causing the computer, via a data-communications medium,to interactively deliver to the client the formed battery of tests, andto accept from the client a substantially completed response to thedelivered formed battery of tests;

d) tied to third computer readable program code, fourth computerreadable program code for causing the computer to analyze the acceptedresponse; and

e) tied to the fourth computer readable program code, fifth computerreadable program code for causing the computer, into the prior historyof the client to, integrate the accepted substantially completedresponse or at least one analytical metric thereof

The present invention collaterally relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein for, a client side (1010) compliance with asoftware-driven protocol (1002) for managing a virtual clinicalneuro-psychological testing program, the computer readable program codein said article of manufacture including:

a) first computer readable program code for causing the computer to, viaa data-communications medium, interactively, receive by the client, theformed battery of tests;

b) the client to substantially complete the tests; and

c) the client to return the substantially completed response to theformed battery of tests.

The present invention likewise relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein for, clinician side (1009) management of a clinicalprotocol (1003) for normal-use activities, the computer readable programcode in said article of manufacture including:

a) first computer readable program code for causing the computer, forsubstantially each client in an ensemble of clients, to monitor at leastone metric of normal-use activities;

b) tied to the first computer readable program code, second computerreadable program code for causing the computer, for substantially eachmetric of the at least one metric of normal use activities, until apredetermined threshold of validation is achieved for an ensemble ofclients, to first manage a virtual clinical neuro-psychological testingprogram for substantially each client in the ensemble; and to correlateanalytical metrics derived from the neuro-psychological testing programwith the monitored metrics of normal-use activities, thereby to validateat least one of the monitored metrics of normal-use activities as aneuro-psychological metric; and

c) tied to the second computer readable program code, third computerreadable program code for causing the computer, for substantially eachvalidated metric of the at least one metric of normal use activities, tosecond manage a virtual clinical neuro-psychological testing program forat least one client wherein a validated normal-use activity metric isused as a classical testing instrument.

The present invention further relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein, on the client side (1010), for a clinical protocol(1004) for normal-use activities, the computer readable program code insaid article of manufacture including:

a) first computer readable program code for causing the computer, forsubstantially each client in an ensemble of clients, to monitor at leastone metric of normal-use activities by the client;

b) tied to the first computer readable program code, second computerreadable program code for causing the computer, for substantially eachvalidated metric of the at least one metric of normal use activities,for at least one client to perform a virtual clinicalneuro-psychological testing program wherein a validated normal-useactivity metric is used as a classical testing instrument.

The present invention also relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein for a software-driven protocol (1005) for the managinga virtual clinical neuro-psychological dynamic hierarchical testingprogram, on the clinician side (1009), the computer readable programcode in said article of manufacture including for each client of aplurality of clients:

a) first computer readable program code for causing a computer toevaluate a prior history of the client;

b) tied to the first computer readable program code, second computerreadable program code for causing the computer, according to theevaluated prior history, to:

i. interactively form an appropriate battery of tests for testing theclient wherein said battery incorporates pseudo-randomization of atleast one representational or organizational parameter;

ii. via a data-communications medium, interactively deliver to theclient the formed battery of tests,

iii. accept from the client a substantially completed response to thedelivered formed battery of tests;

iv. analyze the accepted response; and

v. return to step b)i until complete; and

c) tied to the second computer readable program code, third computerreadable program code for causing the computer, into the prior historyof the client, to integrate:

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

The present invention furthermore relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein for, a client side (1010) compliance with asoftware-driven protocol (1006) for managing a virtual clinicalneuro-psychological dynamic hierarchical testing program, the computerreadable program code in said article of manufacture, tied to the firstcomputer readable program code, second computer readable program codefor causing the computer, according to the evaluated prior history:

a) via a data-communications medium, the client, interactively toreceive the test program,

b) the client to substantially complete a response to the delivered testprogram, and

c) via a data-communications medium, the client to return thesubstantially complete test program.

The present invention likewise, relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein for a software-driven protocol (1007) for managing avirtual clinical neuro-psychological testing program based on normal useactivities, on the clinician side (1009), the protocol, for each clientof a plurality of clients, the computer readable program code in saidarticle of manufacture including;

a) first computer readable program code for causing a computer toevaluate a prior history of the client;

b) tied to the first computer readable code, second computer readableprogram code for causing the computer to, according to the evaluatedprior history, form at least one normal use activity factor of tests fortesting the client wherein said at least one normal use activity factorincorporates pseudo-randomization of at least one representational ororganizational parameter;

c) tied to the second computer readable code, third computer readableprogram code for causing the computer, via a data-communications medium,to interactively

i. deliver the formed at least one normal use activity factor tests tothe client,

ii. the client substantially complete the normal use activity factortests, and

iii. accept, from the client, a substantially completed response to thedelivered normal use activity factor tests;

d) analyze the accepted response; and

e) integrate into the prior history of the client.

The present invention similarly relates to a computer program productincluding a computer usable medium having computer readable program codeembodied therein for a software-driven protocol (1008) for managing avirtual clinical neuro-psychological testing program based on normal useactivities, on the client side (1010), the protocol, for each client ofa plurality of clients, the computer readable program code in saidarticle of manufacture including:

a) first computer readable program code for causing the computer, via adata-communications medium, to interactively

i. receive the formed at least one normal use activity factor tests bythe client,

ii. the client substantially complete the normal use activity factortests, and

iii. send from the client, the substantially completed response to thedelivered normal use activity factor tests.

FIG. 11 illustrates a schematic view characterizing a program storagedevice (1100) readable by machine, tangibly embodying a program ofinstructions executable by the machine to perform method steps for asoftware driven protocol for managing a software driven protocol formanaging a virtual clinical neuro psychological testing program.

The present invention, in addition, relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for a software-drivenprotocol (1101) for managing a virtual clinical neuro-psychologicaltesting program, the protocol including for each client of a pluralityof clients, on the clinician side, steps including:

a) evaluating of a prior history of the client;

b) according to the evaluated prior history, forming an appropriatebattery of tests for testing the client wherein said batteryincorporates pseudo-randomization of at least one representational ororganizational parameter;

c) via a data-communications medium, interactively

i. delivering, to the client, the formed battery of tests, and

ii. accepting, from the client, a substantially completed response tothe delivered formed battery of tests;

d) analyzing the accepted response; and

e) into the prior history of the client, integrating

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

The present invention additionally relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for a software-drivenprotocol (1102) for managing a virtual clinical neuro-psychologicaltesting program, the protocol including for each client of a pluralityof clients, on the client side, steps including:

a) via a data-communications medium, interactively the client receivingthe formed battery of tests,

b) the client substantially completing the formed battery of tests, and

c) via a data-communications medium, the client sending, a substantiallycompleted response to the delivered formed battery of tests.

The present invention in addition relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for, in the clinicianside, a clinical protocol (1103) for normal-use activities, stepsincluding:

a) for substantially each client in an ensemble of clients, monitoringat least one metric of normal-use activities;

b) for substantially each metric of the at least one metric of normaluse activities, until a predetermined threshold of validation isachieved for an ensemble of clients, first managing a virtual clinicalneuro-psychological testing program for substantially each client in theensemble; and correlating analytical metrics derived from theneuro-psychological testing program with the monitored metrics ofnormal-use activities, thereby validating at least one of the monitoredmetrics of normal-use activities as a neuro-psychological metric; and

c) for substantially each validated metric of the at least one metric ofnormal use activities, second managing a virtual clinicalneuro-psychological testing program for at least one client wherein avalidated normal-use activity metric is used as a classical testinginstrument.

The present invention additionally relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for, in the clientside, a clinical protocol (1104) for normal-use activities, said methodsteps including:

a) for substantially each client in an ensemble of clients, performingat least one metric of normal-use activities;

b) for substantially each metric of the at least one metric of normaluse activities, until a predetermined threshold of validation isachieved for an ensemble of clients, carrying out a virtual clinicalneuro-psychological testing program by substantially each client in theensemble; and

c) for substantially each validated metric of the at least one metric ofnormal use activities, carrying out a virtual clinicalneuro-psychological testing program wherein a validated normal-useactivity metric is used as a classical testing instrument.

Further, the present invention also relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for, on the clinicianside, a software-driven protocol (1105) for managing a virtual clinicalneuro-psychological dynamic hierarchical testing program, the protocolincluding for each client of a plurality of clients the steps:

a) evaluating of a prior history of the client;

b) according to the evaluated prior history,

i. interactively forming an appropriate battery of tests for testing theclient wherein said battery incorporates pseudo-randomization of atleast one representational or organizational parameter;

ii. via a data-communications medium, interactively delivering, to theclient, the formed battery of tests,

iii. accepting, from the client, a substantially completed response tothe delivered formed battery of tests;

iv. analyzing the accepted response; and

v. returning to step b)i until complete; and

c) into the prior history of the client, integrating

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

Moreover the present invention also relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for, on the clientside, a software-driven protocol (1106) for managing a virtual clinicalneuro-psychological dynamic hierarchical testing program, the protocolsaid method including for each client of a plurality of clients thesteps:

a) via a data-communications medium, interactively the client receiving,from the clinician, the formed, dynamic hierarchical testing program;

b) the client substantially carrying out the received tests,

c) via a data-communications medium, the client sending, a substantiallycompleted response to the delivered tests.

In addition the present invention likewise relates to a program storagedevice readable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for, a software-drivenprotocol (1107) for managing a virtual clinical neuro-psychologicaltesting program based on normal use activities, the protocol including,on the clinician side, for each client of a plurality of clients thesaid method steps including:

a) evaluating of a prior history of the client;

b) according to the evaluated prior history, forming at least one normaluse activity factor of tests for testing the client wherein said atleast one normal use activity factor incorporates pseudo-randomizationof at least one representational or organizational parameter;

c) via a data-communications medium, interactively

i. delivering the formed at least one normal use activity factor teststo the client,

ii. the client substantially completing the normal use activity factortests, and

iii. accepting, from the client, a substantially completed response tothe delivered normal use activity factor tests;

d) analyzing the accepted response; and

e) into the prior history of the client, integrating

i. the accepted substantially completed response or

ii. at least one analytical metric thereof.

Finally, the present invention also relates to a program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform method steps for, a software-drivenprotocol (1108) for managing a virtual clinical neuro-psychologicaltesting program based on normal use activities, the protocol including,on the client side, for each client of a plurality of clients the saidmethod steps including:

a) via a data-communications medium, the client, interactively receivingthe formed at least one normal use activity factor tests;

b) the client substantially completing the normal use activity factortests, and

c) via a data-communications medium, the client returning asubstantially completed response to the delivered normal use activityfactor tests.

Having described various large-scale embodiments of the presentinvention, it would seem appropriate to introduce some of the novelworking components, which are incorporated into the various embodiments.

At present, in the field of neuro-psychology, a trained, fully qualifiedexpert should have the opportunity to spend a substantial amount of timewith each client. This time should include a significant amount ofobservation and testing, on and one-on-one basis. Unfortunately, ashortage of fully trained, expert, professionals in this field, hasgiven rise to relatively few clients receiving sufficient testing andadequate treatment. The present invention seeks to address this problem.

It should be understood, at the outset, that a neuro psychologistprofessional has, as a prime objective, a desire to perform as littletesting as possible on each client, to spare clients from some of theinevitable stress and trauma resulting from any testing. Furthermore, ifsuch testing can be largely limited to normal activity testing, thiswould present each client with a minimum of stress and trauma.

Registering a new client: For the facile convenience of the attendingclinician, a template is provided when registering a new client (and,optionally, when a clinician intervenes by evaluating that client'sperformance). The template targets potential functions that may beimportant in an evaluation of the following situations:

1. Normal life activity including aging—even in people who are notthought to have a disease. For example, measuring memory-loss,personality change, confusion, delirium, slowed motor function,decreased dexterity, poor coordination, tremor, driving safety, baselineintelligence, baseline personality profile, etc.

2. High level function. For example, vocational evaluation, securityclearance, etc. Child developmental metrics. For example, language orspeech problems, dyslexia, social problems, behavioral problems,attention deficit hyperactivity disorder (ADHD), poor scholasticperformance, hyperactivity, poor attention, motor delay, lack of or poorcoordination, mental retardation, intellectual impairment,recommendation for medication, special education, speech therapy,occupational therapy, etc.

3. Dementia (decreased functioning associated with mental decline). Forexample, to provide information useful for diagnosis and follow-up ofAlzheimer's disease or other dementia disorders (such as vasculardementia, dementia caused by depression, AIDS, HIV, and other knowncauses); decision to initiate medical treatment; decision to change orterminate medical treatment; determine cognitive or motor side effectsof drugs; recommendation to stop driving; recommendation for livingassistance at home or in an institution; cognitive rehabilitationtherapy; standardized tool to judge efficacy and safety of new orexisting drugs, etc.

4. Parkinsonian syndromes. For example, to provide information usefulfor diagnosis and follow-up of Parkinson's disease, and other conditionsthat result in slowed movement (multiple system atrophy, striatonigraldegeneration, diffuse Lewy body disease, progressive supranuclear palsy,frontal lobe syndromes, and other known causes); decision to initiatemedical treatment; decision to change or terminate medical treatment;determination of cognitive or motor side effects of drugs;recommendation to stop driving; recommendation for living assistance athome or in institution; cognitive, physical, speech, and occupationalrehabilitation therapy; standardized tool to judge efficacy and safetyof new or existing drugs. For surgical candidates, to evaluate exclusionfactors that would indicate that there is excessive risk to the patient;to evaluate effects of the surgery; to provide objective information toguide adjustment of post-surgical medical or deep brain stimulationtherapy (or other therapy, such as adjustment of flow rate from animplanted pump that may deliver medication at pre-programmed rates);etc.

5. Rehabilitation. For example, to evaluate current status and pasttrends in cognitive and motor function, as it impinges on ADLs; todesign focused treatment programs; to evaluate efficacy of treatment andto decide future treatment; to use (versions of) computerized exercisesas treatment modalities, etc.

6. Involuntary movements. For example, to characterize and help diagnoseuncontrolled movement such as rest tremor, postural tremor,physiological tremor, chorea, ballismus, athetosis, tics, dystonia,myoclonus, dysmetria, action tremor, and cerebellar tremor; to evaluateefficacy of treatment and decide regarding future treatment; etc.

7. Stroke. For example, to evaluate effects of a stroke and to trackrehabilitation; to evaluate efficacy of treatment and decide regardingfuture treatment; determine cognitive or motor side effects of drugs;recommendation to stop driving; recommendation for living assistance athome or in institution; cognitive, physical, speech, and occupationalrehabilitation therapy; standardized tool to judge efficacy and safetyof new or existing drugs, etc.

8. Epilepsy. For example, to evaluate cognitive and motor status; toevaluate efficacy of treatment and decide regarding future treatment;determine cognitive or motor side effects of drugs; recommendation forliving assistance at home or in institution; cognitive, physical,speech, and occupational rehabilitation therapy; standardized tool tojudge efficacy and safety of new or existing drugs. For surgicalcandidates, to evaluate exclusion factors that would indicate that thereis excessive risk to the patient; to evaluate effects of the surgery; toprovide objective information to guide adjustment of post-surgicalmedical or deep brain stimulation therapy (or other therapy, such asadjustment of flow rate from an implanted pump that may delivermedication at pre-programmed rates); etc.

9. Multiple sclerosis. For example, to evaluate cognitive and motorstatus; to evaluate efficacy of treatment and decide regarding futuretreatment; determine cognitive or motor side effects of drugs;recommendation for living assistance at home or in institution;cognitive, physical, speech, and occupational rehabilitation therapy;standardized tool to judge efficacy and safety of new or existing drugs;etc.

10. Head trauma. For example, to evaluate cognitive and motor status; toevaluate efficacy of treatment and decide regarding future treatment;determine cognitive or motor side effects of drugs; recommendation forliving assistance at home or in an institution; cognitive, physical,speech, and occupational rehabilitation therapy; standardized tool tojudge efficacy and safety of new or existing drugs; etc.

11. Neuro-toxicology. For example, evaluate effects of chronic alcoholintake; evaluate effects of acute alcohol intoxication; evaluate effectsof illicit drugs; evaluate effects of legal sedative drugs (Valium,etc.); evaluate effects of detoxification programs; recommendation forcognitive, physical or occupational therapy; recommendation to stopdriving; etc.

12. Psychiatric diseases (depression, mania, psychosis, anxiety,obsessive-compulsive disorder, etc). For example, to evaluate cognitiveand motor status; to evaluate efficacy of treatment and decide regardingfuture treatment; determine cognitive or motor side effects of drugs;recommendation for living assistance at home or in institution;cognitive, physical, speech, and occupational rehabilitation therapy;standardized tool to judge efficacy and safety of new or existing drugs;to evaluate psychiatric comorbidity (conditions present in addition toother neurological or psychiatric disease); etc.

13. Conversion disorders and malingering. For example, used as objectivemeasure to aid in the diagnosis of non-organic neurological syndrome;performance is inconsistent with complaints; detect malingering bynegative response bias.

14. Driving ability. For example, on-road testing to determine whetherthe subject is ‘driving while intoxicated’; testing before dispatch;etc.

15. Self-improvement. For example, used by client (with or withoutclinician recommendation) for practice to improve certain mental ormotor skill; the same or similar tests (with parameter randomization) tobe used to assess progress of improved performance; etc.

Turning to FIG. 12 which illustrates a schematic view characterizing asystem overview (1200) of a software driven protocol for managing avirtual clinical neuro psychological testing program. This systemoverview includes a number of steps, namely: Clinician user interface(1201), patient user interface (1202), pretesting input stages (1203),construction of test battery (1204), data acquisition (1205), datatransfer (1206), data analysis for each test (1207), interpretation ofbattery (1208), database (1210), report generation (1209) and clinicianaccess to results and report (1211).

The clinician user interface (1201) procedure is further described inFIG. 13 with regard to pretesting and the choice of a test battery(1300) indicating the clinician (1301) initiating a clinical data screen(1302), choice of test battery options (1303), choosing from a list ofstandard batteries (1305), utilizing the test battery wizard (1304) withregard to both a custom (1306) and a standard test battery (1307)resulting in the clinician's affirmation of choice (1308) to carry onthe testing or to reconsider the choice of test battery options.

In FIG. 14, the stage of data acquisition (1205) described above, isfurther expanded (1400) as including the steps of test battery codedownloading (1401), delivering screening battery (1402), screeningbattery data processing (1403), decision (1404) as to whether client isable to perform the battery and, if not, to terminate testing session(1405). If the client is able to perform the battery, the level ofperformance (1406) is assessed as at a high or at a low level,respectively then resulting in delivery of a high-level (1407) orlow-level (1408) battery and this leads to the raw data (1409) stimuliand responses for each test.

FIG. 15 illustrates a schematic view characterizing the steps of a TestBattery Wizard (1500) for use in discerning the nature of a battery oftests required with regard to a client's needs as defined (1200),regarding construction of a test battery (1204). The steps as indicated(1500) are initiated by preferences for testing collected via the wizard(decision tree) (1501), and followed by determining (1502) if there is aneed for assessment or rehabilitation. If rehabilitation is indicated,then a brain function related rehabilitation battery (1504) is selected.For further assessment testing, a primary criterion is decided (1503)for the type of battery, namely, brain function related battery (1505),system related battery (1506) or disease related battery (1507).

Also indicated is the stage (1207) of data analysis for each testfurther expanded and described in FIG. 16 which illustrates a schematicview characterizing the steps of data analysis (1600) for each testcarried out in accordance with the software driven protocol for managinga virtual clinical neuro psychological testing program. These stepsinclude commencing with raw data stimuli and responses (1601) and a setof performance parameters (1602) per design of each test and continueswith an algorithm step (1603) for data or signal processing to extractperformance parameters (specific for each test) leading to the step(1605) of conversion of raw score to a statistical measure “z-score” foreach output parameter when compared with normative data (1604) from thedata base, and resulting in set of z-scores (1606) for the entire testbattery.

Referring to the stage of interpretation of test battery (1208), this isfurther enlarged upon in FIG. 17, which illustrates a schematic viewcharacterizing the steps of the interpretation of a test battery (1700)carried out in accordance with the software driven protocol for managinga virtual clinical neuro psychological testing program. Thisinterpretation emanates from a set of z-scores (1701) for an entirebattery and information (1702) on the make of the tests in the batteryin terms of the brain function tested. These are applied to an algorithmstep (1703), for comparing statistical results from multiple tests thatoverlap regarding the brain functions tested. The level of difficultyand extent of assessment differs between tests, and then lead to asummary of performance (1704) in terms of brain functions tested (foreach function, whether normal or abnormal, and extent of abnormality).Thereafter, based on the client testing history (1705) from thedatabase, an algorithm step (1706), for incorporating past performanceand to look at trends in the data, provides a summary of brain functionstested (1707), as change over time is personalized for the client.

Technology relating to presentation of test battery and dataacquisition:

1. Visual and auditory presentation of instructions for each test.

2. Computerized visual and auditory stimuli.

3. Computerized questionnaire for the clinician.

4. Computerized questionnaire for the client, parent, or caregiver.

5. Animation of testing scenario, similar to computer games.

6. Client interacts with the testing program via standard or customizedcomputer input devices, including but not limited to keyboard, mouse,joystick, digitizing tablet, touch-pad, “games” controller, customizedkeyboard, microphone, touch-screen.

7. Client normal activity responses including all types of communicationdevices.

8. Testing software records all data in the form of stimuli andresponses.

Depending on the nature of the test, responses may be in the form ofbinary data, ordinal data, discrete data (scalar values such as reactiontime and movement time), or continuous digitized data (a vector ormatrix of data that may represent performance such as movementtrajectory, sequence or key presses, or vocal response inputs), oron-line or off-line derived parameters (data extracted from the raw datasummarizing the key features of the response, such as response accuracy,total movement time, negative response bias index, etc.)

Technology relating to analysis and interpretation of test results:

1. ‘Raw’ data are analyzed to produce output parameters according to thedesign of the test and the nature of the data thereof. This stepsummarizes the data set from each test into a set of relevant numbers.

2. For binary or ordinal data collected via questionnaires or ratingscales, the data analysis consists of weighted combinations ofindividual responses for each subsection of the test. These are thencompared to relevant normative data, resulting in a statisticallyrelevant score (e.g. z-score) for each mental or motor function assessedby the test.

3. In the case of continuous digitized data collected from mouse,joystick, or other input device, data are analyzed by different time andfrequency domain mathematical techniques to extract output performanceparameters relevant to the test. These generally include, but are notlimited to, accuracy, reaction time, movement time, tremor amplitude andfrequency, indices of learning.

4. Performance parameters thus computed are compared to relevantnormative data using statistical methods to determine whether individualoutput parameters are statistically different from reference data, basedon predetermined criteria for significance (e.g. z-score). These arereferred to as test results.

5. Test results from some or all tests in the testing battery arecompared for patterns of convergence or divergence from expected trends.As each test assesses a fixed set of mental or motor functions, and astest batteries are constructed to provide overlap or redundancy betweendifferent tests in the battery, there are predicted patterns ofconvergence of test results that are known to (or intended to) test thesame or similar mental or motor functions. This may utilize statisticalmethodologies, automata, rule-based, neural network, or other methods.

6. Testing battery interpretation is a summary of the mental or motorfunctions tested in the battery; for each an indication of normality orabnormality as compared to relevant reference populations, theconfidence at which the interpretation was reached based onconsistencies or inconsistencies in the test results, and in certaincases the degree of abnormality.

7. Client-specific interpretation is arrived at by comparing testinterpretation to the client's testing history.

8. Recommendations for further testing or for referral to other healthcare evaluation are provided, based on the interpretation. This may bederived from an expert system utilizing other demographic or otherinformation provided by the clinician, client, or caretaker, incombination with the test battery interpretation.

Construction of Test Batteries

Test batteries are constructed to focus on specific diagnostic ortherapeutic issues. Different options are available according to thepreference of the user.

1. Function-related batteries are constructed to comprehensively testone or more brain functions. This may be appropriate for research use orfor when there is a focused clinical question requiring thorough testingof only one or a few areas of brain function.

Technology: Done using a look-up table of attributes of available tests.

2. Symptom-related batteries are constructed to characterize specificsymptoms or complaints, as relevant to diagnosis and clinical follow-up.These are designed to test brain functions that are directly ortangentially related to the symptom, as well as brain functions that mayaid in including or excluding certain disease entities from thedifferential diagnosis.

Technology: Done using an expert system containing knownneuro-psychological deficits from experimental and clinical literature.

3. Disease-related batteries are constructed for use when following aclient with a known disease entity. Here, the batteries are designed totest brain functions that are known to be (or suspected to be) affectedby the disease or its treatment. These may provide useful data to theclinician to determine the stage of the disease, provide prognosticdata, and guide treatment decisions. These are considered essential forevaluating new or experimental therapies for specific disease entitiesor syndromes.

Technology: Done using an expert system containing knownneuro-psychological deficits from experimental and clinical literature.

4. Custom batteries are constructed according to specific questionsposed by the clinician, or suggested by a rule-based system of thepresent invention, which evaluates the client testing history.

Technology: Done using an expert system containing knownneuro-psychological deficits from experimental and clinical literature.

5. Cognitive-Rehabilitation testing batteries are constructed forsituations where the client is a participant in a cognitive or motorrehabilitation program utilizing computerized tests (described below).Here, the testing batteries are chosen automatically by the ‘computer’to be independent of the interactive computer exercises used for theindividualized cognitive rehabilitation program. The test batteries arechosen to assess progress on the cognitive rehabilitation program, basedon both the client's previous testing history and treatment plan andprogress. Therefore, each battery is designed to focus on specific brainfunctions, as applicable to the individual client. This type of testingprovides useful data to the rehabilitation professionals (psychiatrist,occupational therapist, physical therapist, speech therapist, cognitivetherapist, others) caring for the client.

Technology: Done using an expert system containing knownneuro-psychological deficits from experimental and clinical literature.

6. Self-improvement batteries are constructed similar to thefunction-related batteries, except to provide for the purpose of theclient to practice on certain cognitive or motor skills. Here, the testswill consist of those with sufficiently steep learning curves,randomize-able parameters, and multiple levels of difficulty.

Choice of Test Batteries

The clinician may choose appropriate test batteries with the help of anembodiment of a rule base system of the present invention:

1. Choose from a list of standard batteries. A list of batteries can beaccessed according to an index, categorized by type of use.Subcategories include symptom and disease-related batteries, eachcontaining a description of the test including length of test, brainfunctions tested, intended use, level of difficulty.

2. Test battery ‘wizard’ to guide clinician in choice of battery, withrecommendation based on client testing history. This consists of adecision-tree algorithm that poses questions to the user, and proceedsaccording to choice of an answer from a menu. At each step, the userverifies the choice.

a) Alternatively, a free data entry field is available for the user toenter his answer. An algorithm will search from a list of keyword forthe best fit (like Mesh headings from Grateful Med).

b) Eventually, the algorithm suggests the appropriate test battery thatbest satisfies the criteria requested by the user.

c) If the user's criteria are not satisfied by the available batteries,or if the user refuses the batteries suggested by the wizard, then anoption is given for formation of a custom battery. The wizard maysuggest the need for formation of a custom test battery.

3. Formation of custom test battery:

a) According to criteria posed by the user in the ‘wizard’.

b) According to results of client's previous testing history.

Algorithm for hierarchical testing (within each testing battery chosenas above)

Once an appropriate battery is selected, a short screening battery ispresented. Based on the results of the screening battery, the followingpaths might be automatically chosen:

1. If the client is unable to perform the tests because of insufficientlevel of arousal, attention capacity, or language skills, then thetesting is terminated.

2. If the results indicate high level of performance, then a detailedhigh-level testing battery is presented.

3. If the results indicate low level of performance, then a detailedlow-level testing battery is presented.

4. Note: for any given test battery the difference between thehigh-level and low-level options may be different tests (as some testsare geared either for high or low ability), or the same test set fordifferent levels of difficulty.

5. Each testing battery focuses on a small set of brain functions, butscreens additional brain functions. The interpretation for the batteryincludes recommendation for further in-depth testing of those screenedbrain functions that were found abnormal.

Management of Test Batteries

1. Test batteries are stored in a database. The database is referencedby multiple fields, including but not limited to: name of test, categoryof test, symptoms, diseases, intended uses, brain functions tested.

2. As custom batteries are built, they are added to the test batterydatabase.

Technology relating to dynamic development of normative values based onlarge body of data:

1. Data from each test is added to the database for that test, storedwith user-input clinical data; including age, level of difficulty,clinical condition (based on client input; e.g. normal, known disease,suspected disease), special circumstances (e.g. Parkinsonian ON/OFFstate, time of day), etc.

Technology: as established for developing norms for psychological tests;

Special attention to method for adjusting for

a) level of difficulty in the test

b) randomized factors

Reliability of Clinical Data

a) Judged reliable—added to database

b) Judged unreliable—not added

Technology: methods for checking internal consistency and sufficientuser input

2. For scoring (interpretation) of tests: appropriate reference normsare compiled from database, and z-score is computed.

Technology: application of statistical models, such as linear, multiple,or logistic data regression may be appropriate.

Test parameter randomization—enhancing validity of repeated testing

For certain tests, whenever possible and appropriate, some non-essentialtest parameters are varied in a pseudo-random fashion:

1. Essential parameters are defined as those that directly relate to thebrain functions being assessed by the test.

2. Non-essential parameters are defined as those that may be changedwithout affecting the brain functions being tested.

Illustration—In a test of visual acuity the essential parameter is thesize of the characters on each line. A non-essential parameter is theactual sequence of characters on each line. It is either assumed orknown that changing the non-essential parameter will not affect theassessment of visual acuity. Rather, the randomization of this parameterenhances validity of test repetition by removing practice effects or theability of the client to memorize the sequence from trial to trial.

Technology: set of non-essential parameters determined per test design,with “expert” input. For repeat client testing, previous choices will beremoved from set, and then random (or pseudo-random) choice of parametervalue is applied to the present test.

What is claimed is:
 1. A software driven protocol for managing a virtualclinical neuro-psychological testing program, the protocol including foreach client of a plurality of clients the steps: a) evaluating of aprior history of the client; b) according to the evaluated priorhistory, forming an appropriate battery of tests for testing the clientwherein said battery incorporates pseudo-randomization of at least onerepresentational or organizational parameter; c) via adata-communications medium, interactively i. delivering, to the client,the formed battery of tests, and ii. accepting, from the client, asubstantially completed response to the delivered formed battery oftests; d) analyzing the accepted response; and e) into the prior historyof the client, integrating i. the accepted substantially completedresponse or ii. at least one analytical metric thereof.
 2. The protocolaccording to claim 1 wherein forming an appropriate battery of testsincludes choosing at least two tests for measuring at least one mentalor motor function, and the tests are selected from the list of testinginstruments for measuring: mental health, intelligence, IQ, memory,immediate recall, memory encoding, memory retrieval, working memory,semantic memory, procedural learning, sequence learning, conditionedresponse, Pavlovian learning, associative learning, implicit learning,explicit learning, block learning, motor learning, pattern matching,judgment, attention, concentration, visual-spatial perception, velocityperception, distance perception, visual searching, calculationalability, mathematical ability, abstract thinking, symbolic thinking,adaptation, sensory-motor adaptation, language, reading, naming,comprehension, classification, direction-following, vigilance, motor,sensory-motor, coordination, psychomotor performance, dexterity, motorskills, tremor, physiological tremor, simple reaction time, choicereaction time, sustained attention, selected attention, dividedattention, driving safety, ballistic movement, bradykinesia,hypokinesia, akinesia, hypometria, movement speed, movement smoothness,movement accuracy, repetitive movement, accurately timed movements,bimanual coordination, hand-eye coordination, personality, scholasticperformance, depression, psychosis, neurosis, anxiety, stress,post-traumatic stress, dementia, static visual acuity, dynamic visualacuity, handwriting analysis, speech analysis, voice tremor, or metricsof interacting with a testing instrument.
 3. The protocol according toclaim 1 wherein forming includes for at least one test in the battery oftests defining a subset of equivalent validation of testing objects andtherein pseudo randomization includes randomizing amongst thesubstantially validated testing objects.
 4. The protocol according toclaim 1 wherein evaluating includes a disclosure of information by theclient.
 5. The protocol according to claim 1 wherein evaluating of aprior history of the client includes: interactively merging practitionerrecommendations for this client into the prior history.
 6. The protocolaccording to claim 1 further including the step of scheduling a nextevaluating of a prior history of the client.
 7. The protocol accordingto claim 1 wherein analyzing the accepted response includes: calculatingan analytical metric of client performance using the accepted responsein the formed battery of tests of at least one client performanceparameter from the accepted response of a test in the formed battery oftests.
 8. The protocol according to claim 1 wherein analyzing theaccepted response includes: calculating an analytical metric of clientperformance using the accepted response in the formed battery of testsof at least one convergence parameter from the accepted response of atleast two tests in the formed battery of tests.
 9. The protocolaccording to claim 1 wherein analyzing the accepted response includesfor a client: applying a rule-based criteria for quantifying at leastone difference between a pair of metrics selected from the list: a) theevaluated prior history of the client; b) a metric of expectedperformance on at least one test of the formed battery of tests; c)actual performance on at least one test of the formed battery of tests;and d) normative values based on large body of data for at least onetest of the formed battery of tests.
 10. The protocol according to claim1 wherein a test of the delivered and accepted formed battery of testsincludes a metric of the clients interaction with a client's peripheraldevice, and the device is selected from the list: Color graphic display;B/W graphic display; Audio speaker; Audio stereophonic headphones;Mouse; Joystick; Roller-ball; Keyboard; Galvanic skin response monitor;Web-cam camera; Microphone; personal communication device, Touch pad; orTouch screen.
 11. A clinical protocol for normal-use activities, theprotocol including the steps of: a) for substantially each client in anensemble of clients, monitoring at least one metric of normal-useactivities; b) for substantially each metric of the at least one metricof normal use activities, until a predetermined threshold of validationis achieved for an ensemble of clients, first managing a virtualclinical neuro-psychological testing program for substantially eachclient in the ensemble; and correlating analytical metrics derived fromthe neuro-psychological testing program with the monitored metrics ofnormal-use activities, thereby validating at least one of the monitoredmetrics of normal-use activities as a neuro-psychological metric; and c)for substantially each validated metric of the at least one metric ofnormal use activities, second managing a virtual clinicalneuro-psychological testing program for at least one client wherein avalidated normal-use activity metric is used as a classical testinginstrument.
 12. The clinical protocol according to claim 11 wherein forsubstantially each client in an ensemble of clients, monitoring at leastone metric of normal-use activities includes downloading a plug-in tothe client's machine, installing the plug-in, or uploading datacollected by the plug-in.
 13. The clinical protocol according to claim11 wherein a metric of normal use activity of the monitored at least onemetric of normal use activities is calculated by measuring an activityselected from the list of copying, printing, pasting, editing,inserting, formatting, exchanging tasks, web surfing, backspacing,deleting or shifting.
 14. The clinical protocol according to claim 11wherein for substantially each client in an ensemble of clients,monitoring at least one metric of normal-use activities includes,copying, pasting, editing, inserting, formatting, exchanging tasks, websurfing, backspacing, deleting or shifting.
 15. The clinical protocolaccording to claim 11 wherein for substantially each client in anensemble of clients, monitoring at least one metric of normal-useactivities includes computer related interactivity, wireless activity,voice prints frequency analysis, mouse tremor, Geographic PositioningSystem tremor, key strokes, or special case use.
 16. The clinicalprotocol according to claim 11 wherein that first managing and secondmanaging include the steps: a) evaluating a prior history of the client;b) according to the evaluated prior history, forming an appropriatebattery of tests wherein said battery incorporates pseudo-randomizationof at least one representational or organizational parameter; c) via adata communications system, interactively i. delivering, to the client,the formed battery of tests, and ii. accepting, from the client, asubstantially completed response to the delivered formed battery oftests; d) analyzing the accepted response; and e) into the prior testinghistory of the client, integrating i. the accepted substantiallycompleted response or ii. at least one analytical metric thereof. 17.The protocol according to claim 16 wherein forming an appropriatebattery of tests includes choosing at least two tests for measuring atleast one mental or motor function, and the tests are selected from thelist of testing instruments for measuring: mental health, intelligence,IQ, memory, immediate recall, memory encoding, memory retrieval, workingmemory, semantic memory, procedural learning, sequence learning,conditioned response, Pavlovian learning, associative learning, implicitlearning, explicit learning, block learning, motor learning, patternmatching, judgment, attention, concentration, visual-spatial perception,velocity perception, distance perception, visual searching,calculational ability, mathematical ability, abstract thinking, symbolicthinking, adaptation, sensory-motor adaptation, language, reading,naming, comprehension, classification, direction-following, vigilance,motor, sensory-motor, coordination, psychomotor performance, dexterity,motor skills, tremor, physiological tremor, simple reaction time, choicereaction time, sustained attention, selected attention, dividedattention, driving safety, ballistic movement, bradykinesia,hypokinesia, akinesia, hypometria, movement speed, movement smoothness,movement accuracy, repetitive movement, accurately timed movements,bimanual coordination, hand-eye coordination, personality, scholasticperformance, depression, psychosis, neurosis, anxiety, stress,post-traumatic stress, dementia, static visual acuity, dynamic visualacuity, handwriting analysis, speech analysis, voice tremor, or metricsof interacting with a testing instrument.
 18. The protocol according toclaim 16 wherein forming includes for at least one test in the batteryof tests defining a subset of equivalent validation of testing objectsand therein pseudo randomization includes randomizing amongst thesubstantially validated testing objects.
 19. The protocol according toclaim 16 wherein evaluating includes a disclosure of information by theclient.
 20. The protocol according to claim 16 wherein evaluating of aprior history of the client includes: interactively merging practitionerrecommendations for this client into the prior history.
 21. The protocolaccording to claim 16 further including the step of scheduling a nextevaluating of a prior history of the client.
 22. The protocol accordingto claim 16 wherein analyzing the accepted response includes:calculating an analytical metric of client's performance using theaccepted response in the formed battery of tests of at least one clientperformance parameter from the accepted response of a test in the formedbattery of tests.
 23. The protocol according to claim 16 whereinanalyzing the accepted response includes: calculating an analyticalmetric of client's performance using the accepted response in the formedbattery of tests of at least one convergence parameter from the acceptedresponse of at least two tests in the formed battery of tests.
 24. Theprotocol according to claim 16 wherein analyzing the accepted responseincludes for a client: applying a rule-based criteria for quantifying atleast one difference between a pair of metrics selected from the list:a) the evaluated prior history of the client; b) a metric of expectedperformance on at least one test of the formed battery of tests; c)actual performance on at least one test of the formed battery of tests;and d) normative values based on large body of data for at least onetest of the formed battery of tests.
 25. The protocol according to claim16 wherein a test of the delivered and accepted formed battery of testsincludes a metric of the clients interaction with a client's peripheraldevice, and the device is selected from the list: Color graphic display;B/W graphic display; Audio speaker; Audio stereophonic headphones;Mouse; Joystick; Roller-ball; Keyboard; Galvanic skin response monitor;Web-cam camera; Microphone; personal communication device, Touch pad; orTouch screen.
 26. The clinical protocol according to claim 11 whereinfirst managing and second managing include the steps: a) evaluating aprior history of the client; b) according to the evaluated priorhistory, forming an appropriate battery of tests for testing the clientwherein said battery incorporates pseudo-randomization of at least onerepresentational or organizational parameter; c) via adata-communications medium, interactively i. delivering, to the client,the formed battery of tests, and ii. accepting, from the client, asubstantially completed response to the delivered formed battery oftests; d) analyzing the accepted response; and e) into the prior historyof the client, integrating i. the accepted substantially completedresponse or ii. at least one analytical metric thereof.
 27. A softwaredriven protocol for managing a virtual clinical neuro-psychologicaldynamic hierarchical testing program, the protocol including for eachclient of a plurality of clients the steps: a) evaluating of a priorhistory of the client; b) according to the evaluated prior history, i.interactively forming an appropriate battery of tests for testing theclient wherein said battery incorporates pseudo-randomization of atleast one representational or organizational parameter, ii. via adata-communications medium, interactively delivering, to the client, theformed battery of tests, iii. accepting, from the client, asubstantially completed response to the delivered formed battery oftests, iv. analyzing the accepted response, and v. returning to step b)iuntil complete; and c) into the prior history of the client, integratingi. the accepted substantially completed response or ii. at least oneanalytical metric thereof.
 28. The protocol according to claim 27wherein forming an appropriate battery of tests includes choosing atleast two tests for measuring at least one mental or motor function, andthe tests are selected from the list of testing instruments formeasuring: mental health, intelligence, IQ, memory, immediate recall,memory encoding, memory retrieval, working memory, semantic memory,procedural learning, sequence learning, conditioned response, Pavlovianlearning, associative learning, implicit learning, explicit learning,block learning, motor learning, pattern matching, judgment, attention,concentration, visual-spatial perception, velocity perception, distanceperception, visual searching, calculational ability, mathematicalability, abstract thinking, symbolic thinking, adaptation, sensory-motoradaptation, language, reading, naming, comprehension, classification,direction-following, vigilance, motor, sensory-motor, coordination,psychomotor performance, dexterity, motor skills, tremor, physiologicaltremor, simple reaction time, choice reaction time, sustained attention,selected attention, divided attention, driving safety, ballisticmovement, bradykinesia, hypokinesia, akinesia, hypometria, movementspeed, movement smoothness, movement accuracy, repetitive movement,accurately timed movements, bimanual coordination, hand-eyecoordination, personality, scholastic performance, depression,psychosis, neurosis, anxiety, stress, post-traumatic stress, dementia,static visual acuity, dynamic visual acuity, handwriting analysis,speech analysis, voice tremor, or metrics of interacting with a testinginstrument.
 29. The protocol according to claim 27 wherein formingincludes for at least one test in the battery of tests defining a subsetof equivalent validation of testing objects and therein pseudorandomization includes randomizing amongst the substantially validatedtesting objects.
 30. The protocol according to claim 27 whereinevaluating includes a disclosure of information by the client.
 31. Theprotocol according to claim 27 wherein evaluating of a prior history ofthe client includes: interactively merging practitioner recommendationsfor this client into the prior history.
 32. The protocol according toclaim 27 further including the step of scheduling a next evaluating of aprior history of the client.
 33. The protocol according to claim 27wherein analyzing the accepted response includes: calculating ananalytical metric of client performance using the accepted response inthe formed battery of tests of at least one client performance parameterfrom the accepted response of a test in the formed battery of tests. 34.The protocol according to claim 27 wherein analyzing the acceptedresponse includes: calculating an analytical metric of clientperformance using the accepted response in the formed battery of testsof at least one of at least one convergence parameter from the acceptedresponse of at least two tests in the formed battery of tests.
 35. Theprotocol according to claim 27 wherein analyzing the accepted responseincludes for a client, applying a rule-based criteria for quantifying atleast one difference between a pair of metrics selected from the list:a) the evaluated prior history of the client; b) a metric of expectedperformance on at least one test of the formed battery of tests; c)actual performance on at least one test of the formed battery of tests;and d) normative values based on large body of data for at least onetest of the formed battery of tests.
 36. The protocol according to claim27 wherein a test of the delivered and accepted formed battery of testsincludes a metric of the clients interaction with a client's peripheraldevice, and the device is selected from the list: Color graphic display;B/W graphic display; Audio speaker; Audio stereophonic headphones;Mouse; Joystick; Roller-ball; Keyboard; Galvanic skin response monitor;Web-cam camera; Microphone; personal communications device, Touch pad;or Touch screen.
 37. The protocol according to claim 27 further includesthe steps of: a) establishing, upgrading and maintaining as appropriate,a validity metric for a substantially non-validated test, for eachclient of an ensemble of clients; b) introducing a substantiallynon-validated test into the formed battery of tests; c) analyzingresponses to the substantially non-validated tests; and d) correlatingresponses with the responses from the other tests in the formed batteryof tests.
 38. The clinical protocol according to claim 37, whereinestablishing, upgrading and maintaining include: a) for substantiallyeach client in an ensemble of clients, monitoring at least one metric ofnormal-use activities; b) for substantially each metric of the at leastone metric of normal-use activities, until a predetermined threshold ofvalidation is achieved for an ensemble of clients, first managing avirtual clinical neuro-psychological testing program for substantiallyeach client in the ensemble; and correlating analytical metrics derivedfrom the neuro-psychological testing program with the monitored metricsof normal-use activities, thereby validating at least one of themonitored metrics of normal-use activities as a neuro-psychologicalmetric; and c) for substantially each validated metric of the at leastone metric of normal-use activities, second managing a virtual clinicalneuro-psychological testing program for at least one client wherein avalidated normal-use activity metric is used as a classical testinginstrument.
 39. The clinical protocol according to claim 38 wherein forsubstantially each client in an ensemble of clients, monitoring at leastone metric of normal-use activities includes downloading a plug-in tothe client's machine, installing plug-ins, uploading data collected bythe plug-in.
 40. The clinical protocol according to claim 38 wherein ametric of normal use activity of the monitored at least one metric ofnormal use activities is calculated by measuring an activity selectedfrom the list of copying, printing, pasting, editing, inserting,formatting, exchanging tasks, web surfing, backspacing, deleting andshifting.
 41. The clinical protocol according to claim 38 wherein forsubstantially each client in an ensemble of clients, monitoring at leastone metric of normal-use activities includes copying, pasting, editing,inserting, formatting, backspacing, deleting, shifting and web surfing.42. The clinical protocol according to claim 38 wherein forsubstantially each client in an ensemble of clients, monitoring at leastone metric of normal-use activities includes computer relatedinteractivity, wireless activity, voice prints frequency analysis, mousetremor, Geographic Positioning System tremor, key strokes, and specialcase use.
 43. The clinical protocol according to claim 38 wherein thatfirst managing and second managing include the steps: a) evaluating aprior history of the client; b) according to the evaluated priorhistory, forming an appropriate battery of tests for testing the clientwherein said battery incorporates pseudo-randomization of at least onerepresentational or organizational parameter; c) via adata-communications medium, interactively i. delivering, to the client,the formed battery of tests, and ii. accepting, from the client, asubstantially completed response to the delivered formed battery oftests; d) analyzing the accepted response; and e) into the prior historyof the client, integrating i. the accepted substantially completedresponse or ii. at least one analytical metric thereof.
 44. A softwaredriven protocol for managing a virtual clinical neuro-psychologicaltesting program based on normal use activities, the protocol includingfor each client of a plurality of clients the steps: a) evaluating of aprior history of the client; b) according to the evaluated priorhistory, forming at least one normal use activity factor of tests fortesting the client wherein said at least one normal use activity factorincorporates pseudo-randomization of at least one representational ororganizational parameter; c) via a data-communications medium,interactively i. delivering the formed at least one normal use activityfactor tests to the client, ii. the client substantially completing thenormal use activity factor tests, and iii. accepting, from the client, asubstantially completed response to the delivered normal use activityfactor tests; d) analyzing the accepted response; and e) into the priorhistory of the client, integrating i. the accepted substantiallycompleted response or ii. at least one analytical metric thereof. 45.The protocol according to claim 44 wherein forming an appropriatebattery of tests includes choosing at least two tests for measuring atleast one mental or motor function, and the tests are selected from thelist of testing instruments for measuring: mental health, intelligence,IQ, memory, immediate recall, memory encoding, memory retrieval, workingmemory, semantic memory, procedural learning, sequence learning,conditioned response, Pavlovian learning, associative learning, implicitlearning, explicit learning, block learning, motor learning, patternmatching, judgment, attention, concentration, visual-spatial perception,velocity perception, distance perception, visual searching,calculational ability, mathematical ability, abstract thinking, symbolicthinking, adaptation, sensory-motor adaptation, language, reading,naming, comprehension, classification, direction-following, vigilance,motor, sensory-motor, coordination, psychomotor performance, dexterity,motor skills, tremor, physiological tremor, simple reaction time, choicereaction time, sustained attention, selected attention, dividedattention, driving safety, ballistic movement, bradykinesia,hypokinesia, akinesia, hypometria, movement speed, movement smoothness,movement accuracy, repetitive movement, accurately timed movements,bimanual coordination, hand-eye coordination, personality, scholasticperformance, depression, dynamic visual acuity, handwriting analysis,speech analysis, voice tremor, or metrics of interacting with a testinginstrument.
 46. The protocol according to claim 44 wherein formingincludes for at least one test in the battery of tests defining a subsetof equivalent validation of testing objects and therein pseudorandomization includes randomizing amongst the substantially validatedtesting objects.
 47. The protocol according to claim 44 whereinevaluating includes a disclosure of information by the client.
 48. Theprotocol according to claim 44 wherein evaluating of a prior history ofthe client, includes interactively merging practitioner recommendationsfor this client into the prior history.
 49. The protocol according toclaim 44 further including the step of scheduling a next evaluating of aprior history of the client.
 50. The protocol according to claim 44wherein analyzing the accepted response includes: calculating ananalytical metric of client performance using the accepted response inthe formed battery of tests of at least one client performance parameterfrom the accepted response of a test in the formed battery of tests. 51.The protocol according to claim 44 wherein analyzing the acceptedresponse includes: calculating an analytical metric of clientperformance using the accepted response in the formed battery of testsof at least one convergence parameter from the accepted response of atleast two tests in the formed battery of tests.
 52. The protocolaccording to claim 44 wherein analyzing the accepted response includes:applying a rule-based criteria for quantifying at least one differencebetween a pair of metrics selected from the list: a) the evaluated priorhistory of the client; b) the client's expected performance on at leastone test of the formed battery of tests; c) the clients actualperformance on at least one test of the formed battery of tests; and d)normative values based on large body of data for at least one test ofthe formed battery of tests.
 53. The protocol according to claim 44wherein a test of the delivered and accepted formed battery of testsincludes a metric of the clients interaction with a client's peripheraldevice, and the device is selected from the list: Color graphic display;B/W graphic display; Audio speaker; Audio stereophonic headphones;Mouse; Joystick; Roller-ball; Keyboard; Galvanic skin response monitor;Web-cam camera; Microphone; personal communication device; Touch pad; orTouch screen.
 54. The clinical protocol according to claim 44 whereinfor substantially each client in an ensemble of clients, monitoring atleast one metric of normal-use activities includes downloading a plug-into the client's machine, installing plug-ins or uploading data collectedby the plug-in.
 55. The clinical protocol according to claim 44 whereinfor substantially each client in an ensemble of clients, monitoring atleast one metric of normal-use activities includes copying, pasting,editing, inserting, formatting, backspacing, deleting, shifting or websurfing.
 56. The clinical protocol according to claim 44 wherein forsubstantially each client in an ensemble of clients, monitoring at leastone metric of normal-use activities includes computer relatedinteractivity, wireless activity, voice prints frequency analysis, mousetremor, Geographic Positioning System tremor, key strokes, or specialcase use.
 57. The protocol according to claim 27 further includes thesteps of: a) establishing, upgrading and maintaining as appropriate, avalidity metric for a substantially non-validated test, for each clientof an ensemble of clients; b) introducing a substantially non-validatedtest into the formed battery of tests; c) analyzing responses to thesubstantially non-validated tests; and d) correlating responses with theresponses from the other tests in the formed battery of tests.